Add files using upload-large-folder tool

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: transformers
+model_name: Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/1e798q43) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1155/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1155/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1155/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1155/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1155/trainer_state.json

@@ -0,0 +1,297 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1155,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1758758062337024e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1540/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1540/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1540/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1540/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1540/trainer_state.json

@@ -0,0 +1,378 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 1540,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    },
+    {
+      "entropy": 0.5205143204885512,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.663673996925354,
+      "learning_rate": 0.00029078429044885647,
+      "loss": 0.42832801818847654,
+      "mean_token_accuracy": 0.8651104230976584,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.5157249197363853,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.8357914686203003,
+      "learning_rate": 0.00028556108842360404,
+      "loss": 0.4264420700073242,
+      "mean_token_accuracy": 0.8651721149682998,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5092187704145908,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.7227477431297302,
+      "learning_rate": 0.00028009455399339797,
+      "loss": 0.4299094772338867,
+      "mean_token_accuracy": 0.8649037969112396,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5237507157027721,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6605198383331299,
+      "learning_rate": 0.00027439591951750953,
+      "loss": 0.44015567779541015,
+      "mean_token_accuracy": 0.8605978584289551,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5314217877388,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.9264621734619141,
+      "learning_rate": 0.00026847689426267907,
+      "loss": 0.4449501037597656,
+      "mean_token_accuracy": 0.862554576098919,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5156177791953087,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.674964964389801,
+      "learning_rate": 0.0002623496403435053,
+      "loss": 0.4325400161743164,
+      "mean_token_accuracy": 0.8629470297694206,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5290651166439057,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.8422386050224304,
+      "learning_rate": 0.00025602674773234535,
+      "loss": 0.44204349517822267,
+      "mean_token_accuracy": 0.8624594563245773,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5855578513672719,
+      "eval_loss": 0.6782004833221436,
+      "eval_mean_token_accuracy": 0.8134025845390099,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 95.7097,
+      "eval_samples_per_second": 17.313,
+      "eval_steps_per_second": 2.173,
+      "step": 1540
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5658655218065408e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1925/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1925/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1925/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1925/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-1925/trainer_state.json

@@ -0,0 +1,469 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 1925,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    },
+    {
+      "entropy": 0.5205143204885512,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.663673996925354,
+      "learning_rate": 0.00029078429044885647,
+      "loss": 0.42832801818847654,
+      "mean_token_accuracy": 0.8651104230976584,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.5157249197363853,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.8357914686203003,
+      "learning_rate": 0.00028556108842360404,
+      "loss": 0.4264420700073242,
+      "mean_token_accuracy": 0.8651721149682998,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5092187704145908,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.7227477431297302,
+      "learning_rate": 0.00028009455399339797,
+      "loss": 0.4299094772338867,
+      "mean_token_accuracy": 0.8649037969112396,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5237507157027721,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6605198383331299,
+      "learning_rate": 0.00027439591951750953,
+      "loss": 0.44015567779541015,
+      "mean_token_accuracy": 0.8605978584289551,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5314217877388,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.9264621734619141,
+      "learning_rate": 0.00026847689426267907,
+      "loss": 0.4449501037597656,
+      "mean_token_accuracy": 0.862554576098919,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5156177791953087,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.674964964389801,
+      "learning_rate": 0.0002623496403435053,
+      "loss": 0.4325400161743164,
+      "mean_token_accuracy": 0.8629470297694206,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5290651166439057,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.8422386050224304,
+      "learning_rate": 0.00025602674773234535,
+      "loss": 0.44204349517822267,
+      "mean_token_accuracy": 0.8624594563245773,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5855578513672719,
+      "eval_loss": 0.6782004833221436,
+      "eval_mean_token_accuracy": 0.8134025845390099,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 95.7097,
+      "eval_samples_per_second": 17.313,
+      "eval_steps_per_second": 2.173,
+      "step": 1540
+    },
+    {
+      "entropy": 0.513688079526077,
+      "epoch": 4.025990903183885,
+      "grad_norm": 0.71764075756073,
+      "learning_rate": 0.0002495212083900749,
+      "loss": 0.4214688491821289,
+      "mean_token_accuracy": 0.8665371561170223,
+      "num_tokens": 3749929.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.4366507241129875,
+      "epoch": 4.155945419103314,
+      "grad_norm": 0.8995222449302673,
+      "learning_rate": 0.00024284638957086167,
+      "loss": 0.3381219482421875,
+      "mean_token_accuracy": 0.8888532662391663,
+      "num_tokens": 3871403.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.4323717120289803,
+      "epoch": 4.2858999350227425,
+      "grad_norm": 0.7425215840339661,
+      "learning_rate": 0.00023601600635580594,
+      "loss": 0.3422117233276367,
+      "mean_token_accuracy": 0.8885110357403755,
+      "num_tokens": 3990410.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.44047351568937304,
+      "epoch": 4.41585445094217,
+      "grad_norm": 0.7615042328834534,
+      "learning_rate": 0.0002290440934718835,
+      "loss": 0.352070198059082,
+      "mean_token_accuracy": 0.8850394701957702,
+      "num_tokens": 4107377.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.4474208961427212,
+      "epoch": 4.545808966861598,
+      "grad_norm": 0.7616956830024719,
+      "learning_rate": 0.00022194497645409647,
+      "loss": 0.35494945526123045,
+      "mean_token_accuracy": 0.8840466964244843,
+      "num_tokens": 4225238.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4457523064315319,
+      "epoch": 4.675763482781027,
+      "grad_norm": 0.7743670344352722,
+      "learning_rate": 0.00021473324221008624,
+      "loss": 0.35500614166259764,
+      "mean_token_accuracy": 0.8841654139757157,
+      "num_tokens": 4348229.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4438612677156925,
+      "epoch": 4.805717998700455,
+      "grad_norm": 1.1020246744155884,
+      "learning_rate": 0.0002074237090476915,
+      "loss": 0.3541957092285156,
+      "mean_token_accuracy": 0.8849165737628937,
+      "num_tokens": 4471337.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.44938020199537276,
+      "epoch": 4.935672514619883,
+      "grad_norm": 0.8200531601905823,
+      "learning_rate": 0.00020003139622703643,
+      "loss": 0.35912376403808594,
+      "mean_token_accuracy": 0.8818741375207901,
+      "num_tokens": 4596516.0,
+      "step": 1900
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.5258857041883928,
+      "eval_loss": 0.7124063372612,
+      "eval_mean_token_accuracy": 0.8141827643490754,
+      "eval_num_tokens": 4655435.0,
+      "eval_runtime": 95.068,
+      "eval_samples_per_second": 17.43,
+      "eval_steps_per_second": 2.188,
+      "step": 1925
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.9560754550221824e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2310/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2310/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2310/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2310/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2310/trainer_state.json

@@ -0,0 +1,560 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.0,
+  "eval_steps": 500,
+  "global_step": 2310,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    },
+    {
+      "entropy": 0.5205143204885512,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.663673996925354,
+      "learning_rate": 0.00029078429044885647,
+      "loss": 0.42832801818847654,
+      "mean_token_accuracy": 0.8651104230976584,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.5157249197363853,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.8357914686203003,
+      "learning_rate": 0.00028556108842360404,
+      "loss": 0.4264420700073242,
+      "mean_token_accuracy": 0.8651721149682998,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5092187704145908,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.7227477431297302,
+      "learning_rate": 0.00028009455399339797,
+      "loss": 0.4299094772338867,
+      "mean_token_accuracy": 0.8649037969112396,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5237507157027721,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6605198383331299,
+      "learning_rate": 0.00027439591951750953,
+      "loss": 0.44015567779541015,
+      "mean_token_accuracy": 0.8605978584289551,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5314217877388,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.9264621734619141,
+      "learning_rate": 0.00026847689426267907,
+      "loss": 0.4449501037597656,
+      "mean_token_accuracy": 0.862554576098919,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5156177791953087,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.674964964389801,
+      "learning_rate": 0.0002623496403435053,
+      "loss": 0.4325400161743164,
+      "mean_token_accuracy": 0.8629470297694206,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5290651166439057,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.8422386050224304,
+      "learning_rate": 0.00025602674773234535,
+      "loss": 0.44204349517822267,
+      "mean_token_accuracy": 0.8624594563245773,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5855578513672719,
+      "eval_loss": 0.6782004833221436,
+      "eval_mean_token_accuracy": 0.8134025845390099,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 95.7097,
+      "eval_samples_per_second": 17.313,
+      "eval_steps_per_second": 2.173,
+      "step": 1540
+    },
+    {
+      "entropy": 0.513688079526077,
+      "epoch": 4.025990903183885,
+      "grad_norm": 0.71764075756073,
+      "learning_rate": 0.0002495212083900749,
+      "loss": 0.4214688491821289,
+      "mean_token_accuracy": 0.8665371561170223,
+      "num_tokens": 3749929.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.4366507241129875,
+      "epoch": 4.155945419103314,
+      "grad_norm": 0.8995222449302673,
+      "learning_rate": 0.00024284638957086167,
+      "loss": 0.3381219482421875,
+      "mean_token_accuracy": 0.8888532662391663,
+      "num_tokens": 3871403.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.4323717120289803,
+      "epoch": 4.2858999350227425,
+      "grad_norm": 0.7425215840339661,
+      "learning_rate": 0.00023601600635580594,
+      "loss": 0.3422117233276367,
+      "mean_token_accuracy": 0.8885110357403755,
+      "num_tokens": 3990410.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.44047351568937304,
+      "epoch": 4.41585445094217,
+      "grad_norm": 0.7615042328834534,
+      "learning_rate": 0.0002290440934718835,
+      "loss": 0.352070198059082,
+      "mean_token_accuracy": 0.8850394701957702,
+      "num_tokens": 4107377.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.4474208961427212,
+      "epoch": 4.545808966861598,
+      "grad_norm": 0.7616956830024719,
+      "learning_rate": 0.00022194497645409647,
+      "loss": 0.35494945526123045,
+      "mean_token_accuracy": 0.8840466964244843,
+      "num_tokens": 4225238.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4457523064315319,
+      "epoch": 4.675763482781027,
+      "grad_norm": 0.7743670344352722,
+      "learning_rate": 0.00021473324221008624,
+      "loss": 0.35500614166259764,
+      "mean_token_accuracy": 0.8841654139757157,
+      "num_tokens": 4348229.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4438612677156925,
+      "epoch": 4.805717998700455,
+      "grad_norm": 1.1020246744155884,
+      "learning_rate": 0.0002074237090476915,
+      "loss": 0.3541957092285156,
+      "mean_token_accuracy": 0.8849165737628937,
+      "num_tokens": 4471337.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.44938020199537276,
+      "epoch": 4.935672514619883,
+      "grad_norm": 0.8200531601905823,
+      "learning_rate": 0.00020003139622703643,
+      "loss": 0.35912376403808594,
+      "mean_token_accuracy": 0.8818741375207901,
+      "num_tokens": 4596516.0,
+      "step": 1900
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.5258857041883928,
+      "eval_loss": 0.7124063372612,
+      "eval_mean_token_accuracy": 0.8141827643490754,
+      "eval_num_tokens": 4655435.0,
+      "eval_runtime": 95.068,
+      "eval_samples_per_second": 17.43,
+      "eval_steps_per_second": 2.188,
+      "step": 1925
+    },
+    {
+      "entropy": 0.3827864169774942,
+      "epoch": 5.064977257959714,
+      "grad_norm": 0.7284737825393677,
+      "learning_rate": 0.00019257149309971248,
+      "loss": 0.2970101737976074,
+      "mean_token_accuracy": 0.9038334503844755,
+      "num_tokens": 4720113.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.3430165499448776,
+      "epoch": 5.1949317738791425,
+      "grad_norm": 0.8974202275276184,
+      "learning_rate": 0.00018505932789846454,
+      "loss": 0.2502327537536621,
+      "mean_token_accuracy": 0.9151435106992721,
+      "num_tokens": 4842234.0,
+      "step": 2000
+    },
+    {
+      "entropy": 0.3617735780030489,
+      "epoch": 5.32488628979857,
+      "grad_norm": 0.7926186323165894,
+      "learning_rate": 0.00017751033624151158,
+      "loss": 0.25911506652832034,
+      "mean_token_accuracy": 0.9122897359728813,
+      "num_tokens": 4959991.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.3655681725591421,
+      "epoch": 5.454840805717999,
+      "grad_norm": 0.9221174716949463,
+      "learning_rate": 0.00016994002941621663,
+      "loss": 0.2651521873474121,
+      "mean_token_accuracy": 0.9111330035328865,
+      "num_tokens": 5082208.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3543636628985405,
+      "epoch": 5.584795321637427,
+      "grad_norm": 0.7796305418014526,
+      "learning_rate": 0.00016236396250727552,
+      "loss": 0.2593145751953125,
+      "mean_token_accuracy": 0.912629722058773,
+      "num_tokens": 5199699.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3563989091664553,
+      "epoch": 5.714749837556855,
+      "grad_norm": 0.8301284313201904,
+      "learning_rate": 0.00015479770243491373,
+      "loss": 0.26270965576171873,
+      "mean_token_accuracy": 0.9109204006195069,
+      "num_tokens": 5320501.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3632403892278671,
+      "epoch": 5.844704353476283,
+      "grad_norm": 1.0204681158065796,
+      "learning_rate": 0.00014725679596876322,
+      "loss": 0.2649832725524902,
+      "mean_token_accuracy": 0.9091781166195869,
+      "num_tokens": 5442291.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.3748371870070696,
+      "epoch": 5.974658869395712,
+      "grad_norm": 0.9021655917167664,
+      "learning_rate": 0.00013975673778314469,
+      "loss": 0.2670526313781738,
+      "mean_token_accuracy": 0.9082769411802292,
+      "num_tokens": 5560531.0,
+      "step": 2300
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.4775148034095764,
+      "eval_loss": 0.7729372978210449,
+      "eval_mean_token_accuracy": 0.8107542619109154,
+      "eval_num_tokens": 5586522.0,
+      "eval_runtime": 95.0564,
+      "eval_samples_per_second": 17.432,
+      "eval_steps_per_second": 2.188,
+      "step": 2310
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.3465872717188096e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2695/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2695/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2695/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2695/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-2695/trainer_state.json

@@ -0,0 +1,641 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 7.0,
+  "eval_steps": 500,
+  "global_step": 2695,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    },
+    {
+      "entropy": 0.5205143204885512,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.663673996925354,
+      "learning_rate": 0.00029078429044885647,
+      "loss": 0.42832801818847654,
+      "mean_token_accuracy": 0.8651104230976584,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.5157249197363853,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.8357914686203003,
+      "learning_rate": 0.00028556108842360404,
+      "loss": 0.4264420700073242,
+      "mean_token_accuracy": 0.8651721149682998,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5092187704145908,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.7227477431297302,
+      "learning_rate": 0.00028009455399339797,
+      "loss": 0.4299094772338867,
+      "mean_token_accuracy": 0.8649037969112396,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5237507157027721,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6605198383331299,
+      "learning_rate": 0.00027439591951750953,
+      "loss": 0.44015567779541015,
+      "mean_token_accuracy": 0.8605978584289551,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5314217877388,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.9264621734619141,
+      "learning_rate": 0.00026847689426267907,
+      "loss": 0.4449501037597656,
+      "mean_token_accuracy": 0.862554576098919,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5156177791953087,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.674964964389801,
+      "learning_rate": 0.0002623496403435053,
+      "loss": 0.4325400161743164,
+      "mean_token_accuracy": 0.8629470297694206,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5290651166439057,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.8422386050224304,
+      "learning_rate": 0.00025602674773234535,
+      "loss": 0.44204349517822267,
+      "mean_token_accuracy": 0.8624594563245773,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5855578513672719,
+      "eval_loss": 0.6782004833221436,
+      "eval_mean_token_accuracy": 0.8134025845390099,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 95.7097,
+      "eval_samples_per_second": 17.313,
+      "eval_steps_per_second": 2.173,
+      "step": 1540
+    },
+    {
+      "entropy": 0.513688079526077,
+      "epoch": 4.025990903183885,
+      "grad_norm": 0.71764075756073,
+      "learning_rate": 0.0002495212083900749,
+      "loss": 0.4214688491821289,
+      "mean_token_accuracy": 0.8665371561170223,
+      "num_tokens": 3749929.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.4366507241129875,
+      "epoch": 4.155945419103314,
+      "grad_norm": 0.8995222449302673,
+      "learning_rate": 0.00024284638957086167,
+      "loss": 0.3381219482421875,
+      "mean_token_accuracy": 0.8888532662391663,
+      "num_tokens": 3871403.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.4323717120289803,
+      "epoch": 4.2858999350227425,
+      "grad_norm": 0.7425215840339661,
+      "learning_rate": 0.00023601600635580594,
+      "loss": 0.3422117233276367,
+      "mean_token_accuracy": 0.8885110357403755,
+      "num_tokens": 3990410.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.44047351568937304,
+      "epoch": 4.41585445094217,
+      "grad_norm": 0.7615042328834534,
+      "learning_rate": 0.0002290440934718835,
+      "loss": 0.352070198059082,
+      "mean_token_accuracy": 0.8850394701957702,
+      "num_tokens": 4107377.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.4474208961427212,
+      "epoch": 4.545808966861598,
+      "grad_norm": 0.7616956830024719,
+      "learning_rate": 0.00022194497645409647,
+      "loss": 0.35494945526123045,
+      "mean_token_accuracy": 0.8840466964244843,
+      "num_tokens": 4225238.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4457523064315319,
+      "epoch": 4.675763482781027,
+      "grad_norm": 0.7743670344352722,
+      "learning_rate": 0.00021473324221008624,
+      "loss": 0.35500614166259764,
+      "mean_token_accuracy": 0.8841654139757157,
+      "num_tokens": 4348229.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4438612677156925,
+      "epoch": 4.805717998700455,
+      "grad_norm": 1.1020246744155884,
+      "learning_rate": 0.0002074237090476915,
+      "loss": 0.3541957092285156,
+      "mean_token_accuracy": 0.8849165737628937,
+      "num_tokens": 4471337.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.44938020199537276,
+      "epoch": 4.935672514619883,
+      "grad_norm": 0.8200531601905823,
+      "learning_rate": 0.00020003139622703643,
+      "loss": 0.35912376403808594,
+      "mean_token_accuracy": 0.8818741375207901,
+      "num_tokens": 4596516.0,
+      "step": 1900
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.5258857041883928,
+      "eval_loss": 0.7124063372612,
+      "eval_mean_token_accuracy": 0.8141827643490754,
+      "eval_num_tokens": 4655435.0,
+      "eval_runtime": 95.068,
+      "eval_samples_per_second": 17.43,
+      "eval_steps_per_second": 2.188,
+      "step": 1925
+    },
+    {
+      "entropy": 0.3827864169774942,
+      "epoch": 5.064977257959714,
+      "grad_norm": 0.7284737825393677,
+      "learning_rate": 0.00019257149309971248,
+      "loss": 0.2970101737976074,
+      "mean_token_accuracy": 0.9038334503844755,
+      "num_tokens": 4720113.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.3430165499448776,
+      "epoch": 5.1949317738791425,
+      "grad_norm": 0.8974202275276184,
+      "learning_rate": 0.00018505932789846454,
+      "loss": 0.2502327537536621,
+      "mean_token_accuracy": 0.9151435106992721,
+      "num_tokens": 4842234.0,
+      "step": 2000
+    },
+    {
+      "entropy": 0.3617735780030489,
+      "epoch": 5.32488628979857,
+      "grad_norm": 0.7926186323165894,
+      "learning_rate": 0.00017751033624151158,
+      "loss": 0.25911506652832034,
+      "mean_token_accuracy": 0.9122897359728813,
+      "num_tokens": 4959991.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.3655681725591421,
+      "epoch": 5.454840805717999,
+      "grad_norm": 0.9221174716949463,
+      "learning_rate": 0.00016994002941621663,
+      "loss": 0.2651521873474121,
+      "mean_token_accuracy": 0.9111330035328865,
+      "num_tokens": 5082208.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3543636628985405,
+      "epoch": 5.584795321637427,
+      "grad_norm": 0.7796305418014526,
+      "learning_rate": 0.00016236396250727552,
+      "loss": 0.2593145751953125,
+      "mean_token_accuracy": 0.912629722058773,
+      "num_tokens": 5199699.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3563989091664553,
+      "epoch": 5.714749837556855,
+      "grad_norm": 0.8301284313201904,
+      "learning_rate": 0.00015479770243491373,
+      "loss": 0.26270965576171873,
+      "mean_token_accuracy": 0.9109204006195069,
+      "num_tokens": 5320501.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3632403892278671,
+      "epoch": 5.844704353476283,
+      "grad_norm": 1.0204681158065796,
+      "learning_rate": 0.00014725679596876322,
+      "loss": 0.2649832725524902,
+      "mean_token_accuracy": 0.9091781166195869,
+      "num_tokens": 5442291.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.3748371870070696,
+      "epoch": 5.974658869395712,
+      "grad_norm": 0.9021655917167664,
+      "learning_rate": 0.00013975673778314469,
+      "loss": 0.2670526313781738,
+      "mean_token_accuracy": 0.9082769411802292,
+      "num_tokens": 5560531.0,
+      "step": 2300
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.4775148034095764,
+      "eval_loss": 0.7729372978210449,
+      "eval_mean_token_accuracy": 0.8107542619109154,
+      "eval_num_tokens": 5586522.0,
+      "eval_runtime": 95.0564,
+      "eval_samples_per_second": 17.432,
+      "eval_steps_per_second": 2.188,
+      "step": 2310
+    },
+    {
+      "entropy": 0.29565749216319326,
+      "epoch": 6.1039636127355426,
+      "grad_norm": 1.0129460096359253,
+      "learning_rate": 0.00013231293861939197,
+      "loss": 0.18774839401245116,
+      "mean_token_accuracy": 0.9347463528714587,
+      "num_tokens": 5682578.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.2680290696769953,
+      "epoch": 6.23391812865497,
+      "grad_norm": 0.7047284841537476,
+      "learning_rate": 0.00012494069362063769,
+      "loss": 0.1707102584838867,
+      "mean_token_accuracy": 0.9414009821414947,
+      "num_tokens": 5803409.0,
+      "step": 2400
+    },
+    {
+      "entropy": 0.2615845823287964,
+      "epoch": 6.363872644574399,
+      "grad_norm": 0.8583500981330872,
+      "learning_rate": 0.00011765515090412491,
+      "loss": 0.1720401382446289,
+      "mean_token_accuracy": 0.941322917342186,
+      "num_tokens": 5927439.0,
+      "step": 2450
+    },
+    {
+      "entropy": 0.26200685687363146,
+      "epoch": 6.493827160493828,
+      "grad_norm": 0.9596366286277771,
+      "learning_rate": 0.00011047128043561942,
+      "loss": 0.16701702117919923,
+      "mean_token_accuracy": 0.9415252614021301,
+      "num_tokens": 6052633.0,
+      "step": 2500
+    },
+    {
+      "entropy": 0.2842141987383366,
+      "epoch": 6.623781676413255,
+      "grad_norm": 1.1430736780166626,
+      "learning_rate": 0.00010340384326987922,
+      "loss": 0.18171667098999023,
+      "mean_token_accuracy": 0.935710808634758,
+      "num_tokens": 6167503.0,
+      "step": 2550
+    },
+    {
+      "entropy": 0.27876456022262575,
+      "epoch": 6.753736192332683,
+      "grad_norm": 1.0698399543762207,
+      "learning_rate": 9.646736122038319e-05,
+      "loss": 0.17995393753051758,
+      "mean_token_accuracy": 0.9379886907339096,
+      "num_tokens": 6284023.0,
+      "step": 2600
+    },
+    {
+      "entropy": 0.2657020591944456,
+      "epoch": 6.883690708252112,
+      "grad_norm": 0.8577425479888916,
+      "learning_rate": 8.96760870206418e-05,
+      "loss": 0.17407890319824218,
+      "mean_token_accuracy": 0.9403641560673713,
+      "num_tokens": 6408327.0,
+      "step": 2650
+    },
+    {
+      "epoch": 7.0,
+      "eval_entropy": 0.4080844047264411,
+      "eval_loss": 0.8632635474205017,
+      "eval_mean_token_accuracy": 0.8096621922002389,
+      "eval_num_tokens": 6517609.0,
+      "eval_runtime": 95.784,
+      "eval_samples_per_second": 17.299,
+      "eval_steps_per_second": 2.172,
+      "step": 2695
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.737490610388992e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3080/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3850/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3850/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3850/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3850/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-3850/trainer_state.json

@@ -0,0 +1,914 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 10.0,
+  "eval_steps": 500,
+  "global_step": 3850,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    },
+    {
+      "entropy": 0.6279655678487902,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.7196288108825684,
+      "learning_rate": 0.00032263673148877055,
+      "loss": 0.5418233871459961,
+      "mean_token_accuracy": 0.8383135430177852,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5900149047374725,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.6000070571899414,
+      "learning_rate": 0.00031970020327186393,
+      "loss": 0.5065702056884765,
+      "mean_token_accuracy": 0.8456701630353928,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5950760287046433,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.5828987956047058,
+      "learning_rate": 0.0003164501953184404,
+      "loss": 0.5115758514404297,
+      "mean_token_accuracy": 0.8449143546819687,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5921229894459248,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.6523755788803101,
+      "learning_rate": 0.00031289338558094566,
+      "loss": 0.5099533462524414,
+      "mean_token_accuracy": 0.8458523535728455,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5937536461651325,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.6264522075653076,
+      "learning_rate": 0.0003090370824126593,
+      "loss": 0.5148015594482422,
+      "mean_token_accuracy": 0.8433699604868888,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.594799654185772,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.867798388004303,
+      "learning_rate": 0.000304889209550859,
+      "loss": 0.5112411499023437,
+      "mean_token_accuracy": 0.844317550957203,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5839655491709709,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.5512472987174988,
+      "learning_rate": 0.0003004582898355252,
+      "loss": 0.5124132537841797,
+      "mean_token_accuracy": 0.8445275443792343,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6069950518012047,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.6030759215354919,
+      "learning_rate": 0.00029575342769703865,
+      "loss": 0.521760368347168,
+      "mean_token_accuracy": 0.8433170795440674,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6072480179942571,
+      "eval_loss": 0.666907787322998,
+      "eval_mean_token_accuracy": 0.8143598649364251,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 95.3076,
+      "eval_samples_per_second": 17.386,
+      "eval_steps_per_second": 2.182,
+      "step": 1155
+    },
+    {
+      "entropy": 0.5205143204885512,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.663673996925354,
+      "learning_rate": 0.00029078429044885647,
+      "loss": 0.42832801818847654,
+      "mean_token_accuracy": 0.8651104230976584,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.5157249197363853,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.8357914686203003,
+      "learning_rate": 0.00028556108842360404,
+      "loss": 0.4264420700073242,
+      "mean_token_accuracy": 0.8651721149682998,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5092187704145908,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.7227477431297302,
+      "learning_rate": 0.00028009455399339797,
+      "loss": 0.4299094772338867,
+      "mean_token_accuracy": 0.8649037969112396,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5237507157027721,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6605198383331299,
+      "learning_rate": 0.00027439591951750953,
+      "loss": 0.44015567779541015,
+      "mean_token_accuracy": 0.8605978584289551,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5314217877388,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.9264621734619141,
+      "learning_rate": 0.00026847689426267907,
+      "loss": 0.4449501037597656,
+      "mean_token_accuracy": 0.862554576098919,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5156177791953087,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.674964964389801,
+      "learning_rate": 0.0002623496403435053,
+      "loss": 0.4325400161743164,
+      "mean_token_accuracy": 0.8629470297694206,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5290651166439057,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.8422386050224304,
+      "learning_rate": 0.00025602674773234535,
+      "loss": 0.44204349517822267,
+      "mean_token_accuracy": 0.8624594563245773,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5855578513672719,
+      "eval_loss": 0.6782004833221436,
+      "eval_mean_token_accuracy": 0.8134025845390099,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 95.7097,
+      "eval_samples_per_second": 17.313,
+      "eval_steps_per_second": 2.173,
+      "step": 1540
+    },
+    {
+      "entropy": 0.513688079526077,
+      "epoch": 4.025990903183885,
+      "grad_norm": 0.71764075756073,
+      "learning_rate": 0.0002495212083900749,
+      "loss": 0.4214688491821289,
+      "mean_token_accuracy": 0.8665371561170223,
+      "num_tokens": 3749929.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.4366507241129875,
+      "epoch": 4.155945419103314,
+      "grad_norm": 0.8995222449302673,
+      "learning_rate": 0.00024284638957086167,
+      "loss": 0.3381219482421875,
+      "mean_token_accuracy": 0.8888532662391663,
+      "num_tokens": 3871403.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.4323717120289803,
+      "epoch": 4.2858999350227425,
+      "grad_norm": 0.7425215840339661,
+      "learning_rate": 0.00023601600635580594,
+      "loss": 0.3422117233276367,
+      "mean_token_accuracy": 0.8885110357403755,
+      "num_tokens": 3990410.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.44047351568937304,
+      "epoch": 4.41585445094217,
+      "grad_norm": 0.7615042328834534,
+      "learning_rate": 0.0002290440934718835,
+      "loss": 0.352070198059082,
+      "mean_token_accuracy": 0.8850394701957702,
+      "num_tokens": 4107377.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.4474208961427212,
+      "epoch": 4.545808966861598,
+      "grad_norm": 0.7616956830024719,
+      "learning_rate": 0.00022194497645409647,
+      "loss": 0.35494945526123045,
+      "mean_token_accuracy": 0.8840466964244843,
+      "num_tokens": 4225238.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4457523064315319,
+      "epoch": 4.675763482781027,
+      "grad_norm": 0.7743670344352722,
+      "learning_rate": 0.00021473324221008624,
+      "loss": 0.35500614166259764,
+      "mean_token_accuracy": 0.8841654139757157,
+      "num_tokens": 4348229.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4438612677156925,
+      "epoch": 4.805717998700455,
+      "grad_norm": 1.1020246744155884,
+      "learning_rate": 0.0002074237090476915,
+      "loss": 0.3541957092285156,
+      "mean_token_accuracy": 0.8849165737628937,
+      "num_tokens": 4471337.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.44938020199537276,
+      "epoch": 4.935672514619883,
+      "grad_norm": 0.8200531601905823,
+      "learning_rate": 0.00020003139622703643,
+      "loss": 0.35912376403808594,
+      "mean_token_accuracy": 0.8818741375207901,
+      "num_tokens": 4596516.0,
+      "step": 1900
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.5258857041883928,
+      "eval_loss": 0.7124063372612,
+      "eval_mean_token_accuracy": 0.8141827643490754,
+      "eval_num_tokens": 4655435.0,
+      "eval_runtime": 95.068,
+      "eval_samples_per_second": 17.43,
+      "eval_steps_per_second": 2.188,
+      "step": 1925
+    },
+    {
+      "entropy": 0.3827864169774942,
+      "epoch": 5.064977257959714,
+      "grad_norm": 0.7284737825393677,
+      "learning_rate": 0.00019257149309971248,
+      "loss": 0.2970101737976074,
+      "mean_token_accuracy": 0.9038334503844755,
+      "num_tokens": 4720113.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.3430165499448776,
+      "epoch": 5.1949317738791425,
+      "grad_norm": 0.8974202275276184,
+      "learning_rate": 0.00018505932789846454,
+      "loss": 0.2502327537536621,
+      "mean_token_accuracy": 0.9151435106992721,
+      "num_tokens": 4842234.0,
+      "step": 2000
+    },
+    {
+      "entropy": 0.3617735780030489,
+      "epoch": 5.32488628979857,
+      "grad_norm": 0.7926186323165894,
+      "learning_rate": 0.00017751033624151158,
+      "loss": 0.25911506652832034,
+      "mean_token_accuracy": 0.9122897359728813,
+      "num_tokens": 4959991.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.3655681725591421,
+      "epoch": 5.454840805717999,
+      "grad_norm": 0.9221174716949463,
+      "learning_rate": 0.00016994002941621663,
+      "loss": 0.2651521873474121,
+      "mean_token_accuracy": 0.9111330035328865,
+      "num_tokens": 5082208.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3543636628985405,
+      "epoch": 5.584795321637427,
+      "grad_norm": 0.7796305418014526,
+      "learning_rate": 0.00016236396250727552,
+      "loss": 0.2593145751953125,
+      "mean_token_accuracy": 0.912629722058773,
+      "num_tokens": 5199699.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3563989091664553,
+      "epoch": 5.714749837556855,
+      "grad_norm": 0.8301284313201904,
+      "learning_rate": 0.00015479770243491373,
+      "loss": 0.26270965576171873,
+      "mean_token_accuracy": 0.9109204006195069,
+      "num_tokens": 5320501.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3632403892278671,
+      "epoch": 5.844704353476283,
+      "grad_norm": 1.0204681158065796,
+      "learning_rate": 0.00014725679596876322,
+      "loss": 0.2649832725524902,
+      "mean_token_accuracy": 0.9091781166195869,
+      "num_tokens": 5442291.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.3748371870070696,
+      "epoch": 5.974658869395712,
+      "grad_norm": 0.9021655917167664,
+      "learning_rate": 0.00013975673778314469,
+      "loss": 0.2670526313781738,
+      "mean_token_accuracy": 0.9082769411802292,
+      "num_tokens": 5560531.0,
+      "step": 2300
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.4775148034095764,
+      "eval_loss": 0.7729372978210449,
+      "eval_mean_token_accuracy": 0.8107542619109154,
+      "eval_num_tokens": 5586522.0,
+      "eval_runtime": 95.0564,
+      "eval_samples_per_second": 17.432,
+      "eval_steps_per_second": 2.188,
+      "step": 2310
+    },
+    {
+      "entropy": 0.29565749216319326,
+      "epoch": 6.1039636127355426,
+      "grad_norm": 1.0129460096359253,
+      "learning_rate": 0.00013231293861939197,
+      "loss": 0.18774839401245116,
+      "mean_token_accuracy": 0.9347463528714587,
+      "num_tokens": 5682578.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.2680290696769953,
+      "epoch": 6.23391812865497,
+      "grad_norm": 0.7047284841537476,
+      "learning_rate": 0.00012494069362063769,
+      "loss": 0.1707102584838867,
+      "mean_token_accuracy": 0.9414009821414947,
+      "num_tokens": 5803409.0,
+      "step": 2400
+    },
+    {
+      "entropy": 0.2615845823287964,
+      "epoch": 6.363872644574399,
+      "grad_norm": 0.8583500981330872,
+      "learning_rate": 0.00011765515090412491,
+      "loss": 0.1720401382446289,
+      "mean_token_accuracy": 0.941322917342186,
+      "num_tokens": 5927439.0,
+      "step": 2450
+    },
+    {
+      "entropy": 0.26200685687363146,
+      "epoch": 6.493827160493828,
+      "grad_norm": 0.9596366286277771,
+      "learning_rate": 0.00011047128043561942,
+      "loss": 0.16701702117919923,
+      "mean_token_accuracy": 0.9415252614021301,
+      "num_tokens": 6052633.0,
+      "step": 2500
+    },
+    {
+      "entropy": 0.2842141987383366,
+      "epoch": 6.623781676413255,
+      "grad_norm": 1.1430736780166626,
+      "learning_rate": 0.00010340384326987922,
+      "loss": 0.18171667098999023,
+      "mean_token_accuracy": 0.935710808634758,
+      "num_tokens": 6167503.0,
+      "step": 2550
+    },
+    {
+      "entropy": 0.27876456022262575,
+      "epoch": 6.753736192332683,
+      "grad_norm": 1.0698399543762207,
+      "learning_rate": 9.646736122038319e-05,
+      "loss": 0.17995393753051758,
+      "mean_token_accuracy": 0.9379886907339096,
+      "num_tokens": 6284023.0,
+      "step": 2600
+    },
+    {
+      "entropy": 0.2657020591944456,
+      "epoch": 6.883690708252112,
+      "grad_norm": 0.8577425479888916,
+      "learning_rate": 8.96760870206418e-05,
+      "loss": 0.17407890319824218,
+      "mean_token_accuracy": 0.9403641560673713,
+      "num_tokens": 6408327.0,
+      "step": 2650
+    },
+    {
+      "epoch": 7.0,
+      "eval_entropy": 0.4080844047264411,
+      "eval_loss": 0.8632635474205017,
+      "eval_mean_token_accuracy": 0.8096621922002389,
+      "eval_num_tokens": 6517609.0,
+      "eval_runtime": 95.784,
+      "eval_samples_per_second": 17.299,
+      "eval_steps_per_second": 2.172,
+      "step": 2695
+    },
+    {
+      "entropy": 0.25731984889087967,
+      "epoch": 7.012995451591943,
+      "grad_norm": 0.5453973412513733,
+      "learning_rate": 8.304397503839827e-05,
+      "loss": 0.16415910720825194,
+      "mean_token_accuracy": 0.9442648204726789,
+      "num_tokens": 6530430.0,
+      "step": 2700
+    },
+    {
+      "entropy": 0.20041885927319528,
+      "epoch": 7.142949967511371,
+      "grad_norm": 0.817810595035553,
+      "learning_rate": 7.658465260289818e-05,
+      "loss": 0.11065036773681641,
+      "mean_token_accuracy": 0.9624890893697738,
+      "num_tokens": 6648623.0,
+      "step": 2750
+    },
+    {
+      "entropy": 0.19894523780792953,
+      "epoch": 7.272904483430799,
+      "grad_norm": 0.7567524313926697,
+      "learning_rate": 7.031139200413975e-05,
+      "loss": 0.10799540519714355,
+      "mean_token_accuracy": 0.962652114033699,
+      "num_tokens": 6768614.0,
+      "step": 2800
+    },
+    {
+      "entropy": 0.19293844722211362,
+      "epoch": 7.402858999350228,
+      "grad_norm": 0.8500803709030151,
+      "learning_rate": 6.423708322164252e-05,
+      "loss": 0.10800057411193847,
+      "mean_token_accuracy": 0.9634559273719787,
+      "num_tokens": 6892420.0,
+      "step": 2850
+    },
+    {
+      "entropy": 0.2034384235367179,
+      "epoch": 7.532813515269655,
+      "grad_norm": 0.9063189029693604,
+      "learning_rate": 5.837420743876688e-05,
+      "loss": 0.11020708084106445,
+      "mean_token_accuracy": 0.9601405537128449,
+      "num_tokens": 7009857.0,
+      "step": 2900
+    },
+    {
+      "entropy": 0.19702354729175567,
+      "epoch": 7.662768031189084,
+      "grad_norm": 0.8015642166137695,
+      "learning_rate": 5.27348113970081e-05,
+      "loss": 0.10956458091735839,
+      "mean_token_accuracy": 0.9614202988147735,
+      "num_tokens": 7132184.0,
+      "step": 2950
+    },
+    {
+      "entropy": 0.20174269448965787,
+      "epoch": 7.792722547108512,
+      "grad_norm": 0.8129525184631348,
+      "learning_rate": 4.733048264295942e-05,
+      "loss": 0.11115622520446777,
+      "mean_token_accuracy": 0.9607632473111153,
+      "num_tokens": 7250668.0,
+      "step": 3000
+    },
+    {
+      "entropy": 0.20127187445759773,
+      "epoch": 7.92267706302794,
+      "grad_norm": 0.916645884513855,
+      "learning_rate": 4.2172325718805845e-05,
+      "loss": 0.10734798431396485,
+      "mean_token_accuracy": 0.9610647630691528,
+      "num_tokens": 7373911.0,
+      "step": 3050
+    },
+    {
+      "epoch": 8.0,
+      "eval_entropy": 0.35548742294598085,
+      "eval_loss": 0.9974557757377625,
+      "eval_mean_token_accuracy": 0.8071597482149417,
+      "eval_num_tokens": 7448696.0,
+      "eval_runtime": 93.987,
+      "eval_samples_per_second": 17.63,
+      "eval_steps_per_second": 2.213,
+      "step": 3080
+    },
+    {
+      "entropy": 0.18474318536382225,
+      "epoch": 8.05198180636777,
+      "grad_norm": 0.5207017660140991,
+      "learning_rate": 3.727093934527048e-05,
+      "loss": 0.09326278686523437,
+      "mean_token_accuracy": 0.967371942409918,
+      "num_tokens": 7497025.0,
+      "step": 3100
+    },
+    {
+      "entropy": 0.165668417327106,
+      "epoch": 8.1819363222872,
+      "grad_norm": 0.6080297827720642,
+      "learning_rate": 3.263639464389803e-05,
+      "loss": 0.0780036211013794,
+      "mean_token_accuracy": 0.9718483003973961,
+      "num_tokens": 7615828.0,
+      "step": 3150
+    },
+    {
+      "entropy": 0.16921501949429513,
+      "epoch": 8.311890838206628,
+      "grad_norm": 0.48100438714027405,
+      "learning_rate": 2.8278214443421888e-05,
+      "loss": 0.07943209648132324,
+      "mean_token_accuracy": 0.9710959446430206,
+      "num_tokens": 7734498.0,
+      "step": 3200
+    },
+    {
+      "entropy": 0.1627163116633892,
+      "epoch": 8.441845354126055,
+      "grad_norm": 0.5401694774627686,
+      "learning_rate": 2.4205353712736153e-05,
+      "loss": 0.0771342420578003,
+      "mean_token_accuracy": 0.9723158326745033,
+      "num_tokens": 7855780.0,
+      "step": 3250
+    },
+    {
+      "entropy": 0.1762950336188078,
+      "epoch": 8.571799870045485,
+      "grad_norm": 0.48023927211761475,
+      "learning_rate": 2.0426181160676853e-05,
+      "loss": 0.0832386302947998,
+      "mean_token_accuracy": 0.9702270576357841,
+      "num_tokens": 7968890.0,
+      "step": 3300
+    },
+    {
+      "entropy": 0.1601188000291586,
+      "epoch": 8.701754385964913,
+      "grad_norm": 0.46383845806121826,
+      "learning_rate": 1.6948462040421234e-05,
+      "loss": 0.07640301704406738,
+      "mean_token_accuracy": 0.9734132272005082,
+      "num_tokens": 8094439.0,
+      "step": 3350
+    },
+    {
+      "entropy": 0.159210016541183,
+      "epoch": 8.83170890188434,
+      "grad_norm": 0.7575987577438354,
+      "learning_rate": 1.3779342193837082e-05,
+      "loss": 0.07497062206268311,
+      "mean_token_accuracy": 0.9726977890729904,
+      "num_tokens": 8219578.0,
+      "step": 3400
+    },
+    {
+      "entropy": 0.15734629351645707,
+      "epoch": 8.961663417803768,
+      "grad_norm": 0.4772493839263916,
+      "learning_rate": 1.0925333368567306e-05,
+      "loss": 0.076096453666687,
+      "mean_token_accuracy": 0.9726072958111763,
+      "num_tokens": 8345441.0,
+      "step": 3450
+    },
+    {
+      "epoch": 9.0,
+      "eval_entropy": 0.3270084271207452,
+      "eval_loss": 1.1192649602890015,
+      "eval_mean_token_accuracy": 0.8066606197792751,
+      "eval_num_tokens": 8379783.0,
+      "eval_runtime": 95.3174,
+      "eval_samples_per_second": 17.384,
+      "eval_steps_per_second": 2.182,
+      "step": 3465
+    },
+    {
+      "entropy": 0.14795680003399825,
+      "epoch": 9.0909681611436,
+      "grad_norm": 0.418293833732605,
+      "learning_rate": 8.392299838019066e-06,
+      "loss": 0.06981793403625489,
+      "mean_token_accuracy": 0.976432801491052,
+      "num_tokens": 8466213.0,
+      "step": 3500
+    },
+    {
+      "entropy": 0.14579848881810903,
+      "epoch": 9.220922677063028,
+      "grad_norm": 0.5972597002983093,
+      "learning_rate": 6.1854463517507314e-06,
+      "loss": 0.06543442726135254,
+      "mean_token_accuracy": 0.9764857029914856,
+      "num_tokens": 8591232.0,
+      "step": 3550
+    },
+    {
+      "entropy": 0.15288641860708593,
+      "epoch": 9.350877192982455,
+      "grad_norm": 0.5795395374298096,
+      "learning_rate": 4.3093074410147585e-06,
+      "loss": 0.0663717269897461,
+      "mean_token_accuracy": 0.9753753918409348,
+      "num_tokens": 8714639.0,
+      "step": 3600
+    },
+    {
+      "entropy": 0.15911089312285184,
+      "epoch": 9.480831708901885,
+      "grad_norm": 0.358585000038147,
+      "learning_rate": 2.7677381014317833e-06,
+      "loss": 0.06874057292938232,
+      "mean_token_accuracy": 0.974975820183754,
+      "num_tokens": 8833066.0,
+      "step": 3650
+    },
+    {
+      "entropy": 0.15434207927435636,
+      "epoch": 9.610786224821313,
+      "grad_norm": 0.733188271522522,
+      "learning_rate": 1.5639058719397926e-06,
+      "loss": 0.07093849658966064,
+      "mean_token_accuracy": 0.9747754210233688,
+      "num_tokens": 8949213.0,
+      "step": 3700
+    },
+    {
+      "entropy": 0.15123332396149636,
+      "epoch": 9.74074074074074,
+      "grad_norm": 0.40724924206733704,
+      "learning_rate": 7.002843262949894e-07,
+      "loss": 0.06883333206176757,
+      "mean_token_accuracy": 0.9754865205287934,
+      "num_tokens": 9069096.0,
+      "step": 3750
+    },
+    {
+      "entropy": 0.1510397135093808,
+      "epoch": 9.870695256660168,
+      "grad_norm": 0.4793565571308136,
+      "learning_rate": 1.7864799049715843e-07,
+      "loss": 0.06909588813781738,
+      "mean_token_accuracy": 0.9750119921565056,
+      "num_tokens": 9188485.0,
+      "step": 3800
+    },
+    {
+      "entropy": 0.1522918649804053,
+      "epoch": 10.0,
+      "grad_norm": 0.9604567885398865,
+      "learning_rate": 6.869658312007251e-11,
+      "loss": 0.06686938762664794,
+      "mean_token_accuracy": 0.975529134273529,
+      "num_tokens": 9310870.0,
+      "step": 3850
+    },
+    {
+      "epoch": 10.0,
+      "eval_entropy": 0.31789582760001606,
+      "eval_loss": 1.177817702293396,
+      "eval_mean_token_accuracy": 0.8061441314908174,
+      "eval_num_tokens": 9310870.0,
+      "eval_runtime": 95.9273,
+      "eval_samples_per_second": 17.274,
+      "eval_steps_per_second": 2.168,
+      "step": 3850
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.9069477834095616e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-770/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-770/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.08930249338906042,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-770/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-770/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test1/checkpoint-770/trainer_state.json

@@ -0,0 +1,206 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 770,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8437769564986228,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.6047701835632324,
+      "learning_rate": 4.2543894243185634e-05,
+      "loss": 1.6604945373535156,
+      "mean_token_accuracy": 0.6506646998226643,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9294079938530921,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 1.1633784770965576,
+      "learning_rate": 8.595603122602811e-05,
+      "loss": 0.8363616180419922,
+      "mean_token_accuracy": 0.7776120388507843,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8483590838313103,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.8963253498077393,
+      "learning_rate": 0.00012936816820887062,
+      "loss": 0.7476110076904297,
+      "mean_token_accuracy": 0.7931549972295762,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7840312227606774,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.7876676917076111,
+      "learning_rate": 0.00017278030519171308,
+      "loss": 0.6980364990234375,
+      "mean_token_accuracy": 0.8027918112277984,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7967149233818054,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.7976633906364441,
+      "learning_rate": 0.00021619244217455558,
+      "loss": 0.6958013153076172,
+      "mean_token_accuracy": 0.8039823538064956,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7755034640431404,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7086226940155029,
+      "learning_rate": 0.00025960457915739807,
+      "loss": 0.6792864990234375,
+      "mean_token_accuracy": 0.8074422472715378,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7452501457929611,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.6065998673439026,
+      "learning_rate": 0.00030301671614024056,
+      "loss": 0.6627902984619141,
+      "mean_token_accuracy": 0.8118429997563362,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7490686719807295,
+      "eval_loss": 0.718986451625824,
+      "eval_mean_token_accuracy": 0.7978653156986604,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 93.2605,
+      "eval_samples_per_second": 17.767,
+      "eval_steps_per_second": 2.23,
+      "step": 385
+    },
+    {
+      "entropy": 0.7291471499893534,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7773680090904236,
+      "learning_rate": 0.0003342599904174574,
+      "loss": 0.6407540893554687,
+      "mean_token_accuracy": 0.816381629387937,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6813811221718789,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.7393642067909241,
+      "learning_rate": 0.0003339921524880377,
+      "loss": 0.6079624938964844,
+      "mean_token_accuracy": 0.825104131102562,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.682051141411066,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.6833657026290894,
+      "learning_rate": 0.0003333814684455912,
+      "loss": 0.6068630218505859,
+      "mean_token_accuracy": 0.8236467111110687,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6863492175936698,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.6060423851013184,
+      "learning_rate": 0.0003324291930928916,
+      "loss": 0.6073496627807617,
+      "mean_token_accuracy": 0.8244831630587578,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6790362980961799,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.7274985313415527,
+      "learning_rate": 0.00033113728311731064,
+      "loss": 0.5929213714599609,
+      "mean_token_accuracy": 0.8260384351015091,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6581329819560051,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6212530136108398,
+      "learning_rate": 0.00032950839307031545,
+      "loss": 0.5812822341918945,
+      "mean_token_accuracy": 0.8288319337368012,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6893622297048568,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.5525115132331848,
+      "learning_rate": 0.0003275458699130301,
+      "loss": 0.6035601425170899,
+      "mean_token_accuracy": 0.8239563027024269,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6713312044739723,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.746279776096344,
+      "learning_rate": 0.0003252537461390686,
+      "loss": 0.59674560546875,
+      "mean_token_accuracy": 0.8267503470182419,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6752213362890941,
+      "eval_loss": 0.6740940809249878,
+      "eval_mean_token_accuracy": 0.8084349325643136,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.8197,
+      "eval_samples_per_second": 17.475,
+      "eval_steps_per_second": 2.194,
+      "step": 770
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.826295845594112e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: transformers
+model_name: Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/eoqvqgs6) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1155/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1155/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.07386918480460901,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1155/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1155/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1155/trainer_state.json

@@ -0,0 +1,297 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1155,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.5888759356737137,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.1323111057281494,
+      "learning_rate": 3.473456711878286e-05,
+      "loss": 1.4242234802246094,
+      "mean_token_accuracy": 0.6861670060455799,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.8638093116879463,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 0.7135503888130188,
+      "learning_rate": 7.017800295427557e-05,
+      "loss": 0.7731611633300781,
+      "mean_token_accuracy": 0.7900564879179001,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8165761995315551,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.5835468173027039,
+      "learning_rate": 0.0001056214387897683,
+      "loss": 0.7186955261230469,
+      "mean_token_accuracy": 0.7992442399263382,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7586414662003517,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.5721960663795471,
+      "learning_rate": 0.00014106487462526099,
+      "loss": 0.6786507415771484,
+      "mean_token_accuracy": 0.8069057819247246,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7753578001260757,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.6396453380584717,
+      "learning_rate": 0.00017650831046075373,
+      "loss": 0.6813118743896485,
+      "mean_token_accuracy": 0.806209616959095,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7591985522210598,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.577964186668396,
+      "learning_rate": 0.00021195174629624644,
+      "loss": 0.67177490234375,
+      "mean_token_accuracy": 0.8090321889519692,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.731777046918869,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.597808301448822,
+      "learning_rate": 0.00024739518213173913,
+      "loss": 0.6601863861083984,
+      "mean_token_accuracy": 0.8120808112621307,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7157932419616443,
+      "eval_loss": 0.7318932414054871,
+      "eval_mean_token_accuracy": 0.7979064600972029,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 94.7419,
+      "eval_samples_per_second": 17.49,
+      "eval_steps_per_second": 2.195,
+      "step": 385
+    },
+    {
+      "entropy": 0.7265488104005555,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7224313020706177,
+      "learning_rate": 0.00027290346308950203,
+      "loss": 0.6452214050292969,
+      "mean_token_accuracy": 0.8156503406002293,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6715995308756828,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.6290106177330017,
+      "learning_rate": 0.000272684789300892,
+      "loss": 0.6042086791992187,
+      "mean_token_accuracy": 0.8257312250137329,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6747516925632954,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.66269850730896,
+      "learning_rate": 0.00027218620198917387,
+      "loss": 0.6056245803833008,
+      "mean_token_accuracy": 0.8240418726205826,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6872365647554397,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.4894042909145355,
+      "learning_rate": 0.00027140872562641226,
+      "loss": 0.615659523010254,
+      "mean_token_accuracy": 0.8227636247873307,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6712110449373722,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.9492104649543762,
+      "learning_rate": 0.00027035395773182937,
+      "loss": 0.595902099609375,
+      "mean_token_accuracy": 0.8245656326413154,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6576551312208175,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6480705142021179,
+      "learning_rate": 0.00026902406558930333,
+      "loss": 0.5864276885986328,
+      "mean_token_accuracy": 0.8273968940973282,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6850327280163765,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.516665518283844,
+      "learning_rate": 0.0002674217817941425,
+      "loss": 0.6083594512939453,
+      "mean_token_accuracy": 0.8232863625884056,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6665814685821533,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.6762926578521729,
+      "learning_rate": 0.00026555039863828637,
+      "loss": 0.6011666488647461,
+      "mean_token_accuracy": 0.8256330332159996,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6573245783264821,
+      "eval_loss": 0.6852125525474548,
+      "eval_mean_token_accuracy": 0.808370346060166,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.2081,
+      "eval_samples_per_second": 17.589,
+      "eval_steps_per_second": 2.208,
+      "step": 770
+    },
+    {
+      "entropy": 0.612116006301276,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.5827146172523499,
+      "learning_rate": 0.0002634137613454702,
+      "loss": 0.5394676208496094,
+      "mean_token_accuracy": 0.8394461966040146,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5671873143315316,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.523046612739563,
+      "learning_rate": 0.00026101626017025327,
+      "loss": 0.4936346435546875,
+      "mean_token_accuracy": 0.8486950224637986,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5720223182439804,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.47807472944259644,
+      "learning_rate": 0.0002583628213771449,
+      "loss": 0.5041716003417969,
+      "mean_token_accuracy": 0.8464827784895896,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5660848580300808,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.5875942707061768,
+      "learning_rate": 0.0002554588971183651,
+      "loss": 0.4948244094848633,
+      "mean_token_accuracy": 0.8494577088952064,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5680096058547497,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.5312080383300781,
+      "learning_rate": 0.0002523104542310364,
+      "loss": 0.5015496826171875,
+      "mean_token_accuracy": 0.847268882393837,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5735848733782768,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.5523076057434082,
+      "learning_rate": 0.0002489239619768288,
+      "loss": 0.4966643524169922,
+      "mean_token_accuracy": 0.8478639185428619,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5666417135298252,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.4719190299510956,
+      "learning_rate": 0.00024530637874924604,
+      "loss": 0.5055577850341797,
+      "mean_token_accuracy": 0.8468799209594726,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.5885502319037914,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.5237556099891663,
+      "learning_rate": 0.00024146513777587047,
+      "loss": 0.5136178588867187,
+      "mean_token_accuracy": 0.845189718902111,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6404194602599511,
+      "eval_loss": 0.6698319315910339,
+      "eval_mean_token_accuracy": 0.8120474551732724,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 94.1854,
+      "eval_samples_per_second": 17.593,
+      "eval_steps_per_second": 2.208,
+      "step": 1155
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.224542794634834e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1540/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1540/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.07386918480460901,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1540/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1540/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1540/trainer_state.json

@@ -0,0 +1,378 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 1540,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.5888759356737137,
+      "epoch": 0.1299545159194282,
+      "grad_norm": 1.1323111057281494,
+      "learning_rate": 3.473456711878286e-05,
+      "loss": 1.4242234802246094,
+      "mean_token_accuracy": 0.6861670060455799,
+      "num_tokens": 127115.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.8638093116879463,
+      "epoch": 0.2599090318388564,
+      "grad_norm": 0.7135503888130188,
+      "learning_rate": 7.017800295427557e-05,
+      "loss": 0.7731611633300781,
+      "mean_token_accuracy": 0.7900564879179001,
+      "num_tokens": 251796.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8165761995315551,
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.5835468173027039,
+      "learning_rate": 0.0001056214387897683,
+      "loss": 0.7186955261230469,
+      "mean_token_accuracy": 0.7992442399263382,
+      "num_tokens": 366505.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7586414662003517,
+      "epoch": 0.5198180636777128,
+      "grad_norm": 0.5721960663795471,
+      "learning_rate": 0.00014106487462526099,
+      "loss": 0.6786507415771484,
+      "mean_token_accuracy": 0.8069057819247246,
+      "num_tokens": 493150.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7753578001260757,
+      "epoch": 0.649772579597141,
+      "grad_norm": 0.6396453380584717,
+      "learning_rate": 0.00017650831046075373,
+      "loss": 0.6813118743896485,
+      "mean_token_accuracy": 0.806209616959095,
+      "num_tokens": 607197.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7591985522210598,
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.577964186668396,
+      "learning_rate": 0.00021195174629624644,
+      "loss": 0.67177490234375,
+      "mean_token_accuracy": 0.8090321889519692,
+      "num_tokens": 726579.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.731777046918869,
+      "epoch": 0.9096816114359974,
+      "grad_norm": 0.597808301448822,
+      "learning_rate": 0.00024739518213173913,
+      "loss": 0.6601863861083984,
+      "mean_token_accuracy": 0.8120808112621307,
+      "num_tokens": 848944.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7157932419616443,
+      "eval_loss": 0.7318932414054871,
+      "eval_mean_token_accuracy": 0.7979064600972029,
+      "eval_num_tokens": 931087.0,
+      "eval_runtime": 94.7419,
+      "eval_samples_per_second": 17.49,
+      "eval_steps_per_second": 2.195,
+      "step": 385
+    },
+    {
+      "entropy": 0.7265488104005555,
+      "epoch": 1.0389863547758285,
+      "grad_norm": 0.7224313020706177,
+      "learning_rate": 0.00027290346308950203,
+      "loss": 0.6452214050292969,
+      "mean_token_accuracy": 0.8156503406002293,
+      "num_tokens": 968580.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.6715995308756828,
+      "epoch": 1.1689408706952567,
+      "grad_norm": 0.6290106177330017,
+      "learning_rate": 0.000272684789300892,
+      "loss": 0.6042086791992187,
+      "mean_token_accuracy": 0.8257312250137329,
+      "num_tokens": 1090002.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6747516925632954,
+      "epoch": 1.2988953866146848,
+      "grad_norm": 0.66269850730896,
+      "learning_rate": 0.00027218620198917387,
+      "loss": 0.6056245803833008,
+      "mean_token_accuracy": 0.8240418726205826,
+      "num_tokens": 1212161.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6872365647554397,
+      "epoch": 1.428849902534113,
+      "grad_norm": 0.4894042909145355,
+      "learning_rate": 0.00027140872562641226,
+      "loss": 0.615659523010254,
+      "mean_token_accuracy": 0.8227636247873307,
+      "num_tokens": 1332852.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6712110449373722,
+      "epoch": 1.5588044184535412,
+      "grad_norm": 0.9492104649543762,
+      "learning_rate": 0.00027035395773182937,
+      "loss": 0.595902099609375,
+      "mean_token_accuracy": 0.8245656326413154,
+      "num_tokens": 1453569.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6576551312208175,
+      "epoch": 1.6887589343729694,
+      "grad_norm": 0.6480705142021179,
+      "learning_rate": 0.00026902406558930333,
+      "loss": 0.5864276885986328,
+      "mean_token_accuracy": 0.8273968940973282,
+      "num_tokens": 1577152.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6850327280163765,
+      "epoch": 1.8187134502923976,
+      "grad_norm": 0.516665518283844,
+      "learning_rate": 0.0002674217817941425,
+      "loss": 0.6083594512939453,
+      "mean_token_accuracy": 0.8232863625884056,
+      "num_tokens": 1691926.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6665814685821533,
+      "epoch": 1.9486679662118258,
+      "grad_norm": 0.6762926578521729,
+      "learning_rate": 0.00026555039863828637,
+      "loss": 0.6011666488647461,
+      "mean_token_accuracy": 0.8256330332159996,
+      "num_tokens": 1816059.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6573245783264821,
+      "eval_loss": 0.6852125525474548,
+      "eval_mean_token_accuracy": 0.808370346060166,
+      "eval_num_tokens": 1862174.0,
+      "eval_runtime": 94.2081,
+      "eval_samples_per_second": 17.589,
+      "eval_steps_per_second": 2.208,
+      "step": 770
+    },
+    {
+      "entropy": 0.612116006301276,
+      "epoch": 2.077972709551657,
+      "grad_norm": 0.5827146172523499,
+      "learning_rate": 0.0002634137613454702,
+      "loss": 0.5394676208496094,
+      "mean_token_accuracy": 0.8394461966040146,
+      "num_tokens": 1931345.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5671873143315316,
+      "epoch": 2.207927225471085,
+      "grad_norm": 0.523046612739563,
+      "learning_rate": 0.00026101626017025327,
+      "loss": 0.4936346435546875,
+      "mean_token_accuracy": 0.8486950224637986,
+      "num_tokens": 2057087.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.5720223182439804,
+      "epoch": 2.3378817413905133,
+      "grad_norm": 0.47807472944259644,
+      "learning_rate": 0.0002583628213771449,
+      "loss": 0.5041716003417969,
+      "mean_token_accuracy": 0.8464827784895896,
+      "num_tokens": 2179801.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.5660848580300808,
+      "epoch": 2.4678362573099415,
+      "grad_norm": 0.5875942707061768,
+      "learning_rate": 0.0002554588971183651,
+      "loss": 0.4948244094848633,
+      "mean_token_accuracy": 0.8494577088952064,
+      "num_tokens": 2300775.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5680096058547497,
+      "epoch": 2.5977907732293697,
+      "grad_norm": 0.5312080383300781,
+      "learning_rate": 0.0002523104542310364,
+      "loss": 0.5015496826171875,
+      "mean_token_accuracy": 0.847268882393837,
+      "num_tokens": 2425909.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5735848733782768,
+      "epoch": 2.727745289148798,
+      "grad_norm": 0.5523076057434082,
+      "learning_rate": 0.0002489239619768288,
+      "loss": 0.4966643524169922,
+      "mean_token_accuracy": 0.8478639185428619,
+      "num_tokens": 2542506.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5666417135298252,
+      "epoch": 2.857699805068226,
+      "grad_norm": 0.4719190299510956,
+      "learning_rate": 0.00024530637874924604,
+      "loss": 0.5055577850341797,
+      "mean_token_accuracy": 0.8468799209594726,
+      "num_tokens": 2666261.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.5885502319037914,
+      "epoch": 2.9876543209876543,
+      "grad_norm": 0.5237556099891663,
+      "learning_rate": 0.00024146513777587047,
+      "loss": 0.5136178588867187,
+      "mean_token_accuracy": 0.845189718902111,
+      "num_tokens": 2781874.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6404194602599511,
+      "eval_loss": 0.6698319315910339,
+      "eval_mean_token_accuracy": 0.8120474551732724,
+      "eval_num_tokens": 2793261.0,
+      "eval_runtime": 94.1854,
+      "eval_samples_per_second": 17.593,
+      "eval_steps_per_second": 2.208,
+      "step": 1155
+    },
+    {
+      "entropy": 0.48768596628203464,
+      "epoch": 3.116959064327485,
+      "grad_norm": 0.45523950457572937,
+      "learning_rate": 0.0002374081318449413,
+      "loss": 0.4034775924682617,
+      "mean_token_accuracy": 0.8715206759059848,
+      "num_tokens": 2896983.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.4783519075810909,
+      "epoch": 3.246913580246914,
+      "grad_norm": 0.5953325629234314,
+      "learning_rate": 0.0002331436970876517,
+      "loss": 0.39595497131347657,
+      "mean_token_accuracy": 0.8736060819029808,
+      "num_tokens": 3019005.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.4747548791766167,
+      "epoch": 3.3768680961663415,
+      "grad_norm": 0.5665897130966187,
+      "learning_rate": 0.00022868059584948654,
+      "loss": 0.4032122039794922,
+      "mean_token_accuracy": 0.8709388446807861,
+      "num_tokens": 3143415.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.4843991830945015,
+      "epoch": 3.50682261208577,
+      "grad_norm": 0.6089479923248291,
+      "learning_rate": 0.00022402799868579694,
+      "loss": 0.4088896179199219,
+      "mean_token_accuracy": 0.8691128557920456,
+      "num_tokens": 3263845.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.49369323417544364,
+      "epoch": 3.636777128005198,
+      "grad_norm": 0.7913607954978943,
+      "learning_rate": 0.00021919546551860548,
+      "loss": 0.4160056304931641,
+      "mean_token_accuracy": 0.8698329451680183,
+      "num_tokens": 3383117.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.47668287783861163,
+      "epoch": 3.7667316439246266,
+      "grad_norm": 0.5151591897010803,
+      "learning_rate": 0.00021419292599336078,
+      "loss": 0.4031526184082031,
+      "mean_token_accuracy": 0.8698683533072472,
+      "num_tokens": 3509724.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.4885013411939144,
+      "epoch": 3.8966861598440543,
+      "grad_norm": 0.6061839461326599,
+      "learning_rate": 0.0002090306590760024,
+      "loss": 0.4131336212158203,
+      "mean_token_accuracy": 0.8689957037568092,
+      "num_tokens": 3631110.0,
+      "step": 1500
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5588342215006168,
+      "eval_loss": 0.6875180006027222,
+      "eval_mean_token_accuracy": 0.8144597127460517,
+      "eval_num_tokens": 3724348.0,
+      "eval_runtime": 94.8129,
+      "eval_samples_per_second": 17.477,
+      "eval_steps_per_second": 2.194,
+      "step": 1540
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3850,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.6306733516670566e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1925/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

DBCA_original_Estonian/Qwen3.5-4B-Base_original_features_structural_train_original_features_structural_test2/checkpoint-1925/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.07386918480460901,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}