2026-05-07 13:27:47,763 | INFO | Starting SFT fine-tuning job
2026-05-07 13:27:47,763 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:27:47,763 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:27:47,763 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:27:47,763 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:27:47,763 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:27:47,804 | INFO | CUDA available: True
2026-05-07 13:27:47,825 | INFO | GPU: NVIDIA GB10
2026-05-07 13:27:47,826 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:27:47,827 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:27:47,827 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:27:47,827 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:27:49,034 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 13:30:30,504 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:30:34,280 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:30:40,321 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:30:40,717 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:30:40,818 | INFO | Train rows: 25340
2026-05-07 13:30:40,819 | INFO | Eval rows: 6336
2026-05-07 13:30:40,819 | INFO | Sample train example: [{'role': 'system', 'content': 'أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.'}, {'role': 'user', 'content': 'نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟'}, {'role': 'assistant', 'content': 'البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.'}]
2026-05-07 13:30:40,819 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:34:03,570 | INFO | Starting SFT fine-tuning job
2026-05-07 13:34:03,570 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:34:03,570 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:34:03,570 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:34:03,570 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:34:03,570 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:34:03,612 | INFO | CUDA available: True
2026-05-07 13:34:03,631 | INFO | GPU: NVIDIA GB10
2026-05-07 13:34:03,633 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:34:03,633 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:34:03,633 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:34:03,633 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:34:04,862 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 13:36:22,953 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:36:26,760 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:36:28,674 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:36:28,675 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:36:28,676 | INFO | Train rows: 25340
2026-05-07 13:36:28,676 | INFO | Eval rows: 6336
2026-05-07 13:36:28,677 | INFO | Sample train example: [{'role': 'system', 'content': 'أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.'}, {'role': 'user', 'content': 'نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟'}, {'role': 'assistant', 'content': 'البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.'}]
2026-05-07 13:36:28,677 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:41:24,497 | INFO | Starting SFT fine-tuning job
2026-05-07 13:41:24,497 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:41:24,497 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:41:24,497 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:41:24,497 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:41:24,497 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:41:24,548 | INFO | CUDA available: True
2026-05-07 13:41:24,576 | INFO | GPU: NVIDIA GB10
2026-05-07 13:41:24,577 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:41:24,578 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:41:24,578 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:41:24,578 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:41:25,691 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 13:44:02,778 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:44:06,613 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:44:08,694 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:44:08,697 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:44:08,698 | INFO | Train rows: 25340
2026-05-07 13:44:08,699 | INFO | Eval rows: 6336
2026-05-07 13:44:08,699 | INFO | Sample train example: [{'role': 'system', 'content': 'أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.'}, {'role': 'user', 'content': 'نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟'}, {'role': 'assistant', 'content': 'البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.'}]
2026-05-07 13:44:08,699 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:44:08,752 | INFO | Training started
2026-05-07 13:46:02,478 | INFO | Starting SFT fine-tuning job
2026-05-07 13:46:02,479 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:46:02,479 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:46:02,479 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:46:02,479 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:46:02,479 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:46:02,523 | INFO | CUDA available: True
2026-05-07 13:46:02,552 | INFO | GPU: NVIDIA GB10
2026-05-07 13:46:02,554 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:46:02,555 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:46:02,555 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:46:02,555 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:46:03,698 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 13:48:19,475 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:48:23,292 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:48:25,199 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:48:25,202 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:48:25,203 | INFO | Train rows: 25340
2026-05-07 13:48:25,203 | INFO | Eval rows: 6336
2026-05-07 13:48:25,204 | INFO | Sample train example: [{'role': 'system', 'content': 'أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.'}, {'role': 'user', 'content': 'نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟'}, {'role': 'assistant', 'content': 'البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.'}]
2026-05-07 13:48:25,204 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:48:25,244 | INFO | Training started
2026-05-07 13:55:23,761 | INFO | Starting SFT fine-tuning job
2026-05-07 13:55:23,761 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:55:23,761 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:55:23,761 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:55:23,761 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:55:23,761 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:55:23,803 | INFO | CUDA available: True
2026-05-07 13:55:23,826 | INFO | GPU: NVIDIA GB10
2026-05-07 13:55:23,828 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:55:23,829 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:55:23,829 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:55:23,829 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:55:24,929 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 13:58:01,693 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:58:05,420 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:58:07,816 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:58:08,954 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 13:58:09,429 | INFO | Train rows: 25340
2026-05-07 13:58:09,430 | INFO | Eval rows: 6336
2026-05-07 13:58:09,430 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-07 13:58:09,430 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:59:25,638 | INFO | Starting SFT fine-tuning job
2026-05-07 13:59:25,639 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 13:59:25,639 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 13:59:25,639 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 13:59:25,639 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 13:59:25,639 | INFO | Torch version: 2.11.0+cu130
2026-05-07 13:59:25,683 | INFO | CUDA available: True
2026-05-07 13:59:25,709 | INFO | GPU: NVIDIA GB10
2026-05-07 13:59:25,710 | INFO | No Hugging Face token provided; skipping login
2026-05-07 13:59:25,711 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 13:59:25,711 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 13:59:25,711 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 13:59:26,819 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 14:02:05,419 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 14:02:09,265 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 14:02:11,637 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 14:02:11,918 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 14:02:12,159 | INFO | Train rows: 25340
2026-05-07 14:02:12,160 | INFO | Eval rows: 6336
2026-05-07 14:02:12,160 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-07 14:02:12,161 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 14:02:23,459 | INFO | Training started
2026-05-07 14:08:07,243 | INFO | Starting SFT fine-tuning job
2026-05-07 14:08:07,244 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 14:08:07,244 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-07 14:08:07,244 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 14:08:07,244 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 14:08:07,244 | INFO | Torch version: 2.11.0+cu130
2026-05-07 14:08:07,286 | INFO | CUDA available: True
2026-05-07 14:08:07,309 | INFO | GPU: NVIDIA GB10
2026-05-07 14:08:07,499 | INFO | Hugging Face login succeeded
2026-05-07 14:08:07,500 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-07 14:08:07,500 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-07 14:08:07,500 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-07 14:08:08,618 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-07 14:10:19,279 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-07 14:10:23,076 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-07 14:10:25,353 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 14:10:25,621 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-07 14:10:25,861 | INFO | Train rows: 25340
2026-05-07 14:10:25,862 | INFO | Eval rows: 6336
2026-05-07 14:10:25,862 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>نحس بوجيعة في العينين من كثرة غسلان الماعون بالماء السخون؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>البخار الصاعد من الماء السخون ينجم يتعب العين ويسبب احمرار، هذاكا علاش حاول تستعمل ماء دافي مش سخون برشة. اغسل وجهك بماء بارد بعد ما تكمل باش تبرد عينيك.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-07 14:10:25,862 | INFO | No checkpoint found in /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-07 14:10:29,187 | INFO | Training started
2026-05-08 04:09:43,113 | INFO | Training finished
2026-05-08 04:09:43,113 | INFO | Saving model and tokenizer to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-08 04:09:45,243 | INFO | Saved training metrics to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft/training_metrics.json
2026-05-08 04:09:45,243 | INFO | Running preview generation on a Tunisian prompt
2026-05-08 04:09:48,487 | INFO | Preview prompt: عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟
2026-05-08 04:09:48,487 | INFO | Preview output: <|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|START_OF_TURN_TOKEN|><|USER_TOKEN|>عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>ح في في في في في في في في في في<|START_OF_TURN_TOKEN|>
2026-05-22 00:54:42,946 | INFO | Starting SFT fine-tuning job
2026-05-22 00:54:42,946 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 00:54:42,947 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 00:54:42,947 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 00:54:42,947 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 00:54:42,947 | INFO | Torch version: 2.11.0+cu130
2026-05-22 00:54:43,079 | INFO | CUDA available: True
2026-05-22 00:54:43,096 | INFO | GPU: NVIDIA GB10
2026-05-22 00:55:42,965 | INFO | Starting SFT fine-tuning job
2026-05-22 00:55:42,965 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 00:55:42,965 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 00:55:42,965 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 00:55:42,965 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 00:55:42,965 | INFO | Torch version: 2.11.0+cu130
2026-05-22 00:55:43,096 | INFO | CUDA available: True
2026-05-22 00:55:43,113 | INFO | GPU: NVIDIA GB10
2026-05-22 00:55:48,743 | INFO | Starting SFT fine-tuning job
2026-05-22 00:55:48,743 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 00:55:48,743 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 00:55:48,743 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 00:55:48,743 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 00:55:48,743 | INFO | Torch version: 2.11.0+cu130
2026-05-22 00:55:48,878 | INFO | CUDA available: True
2026-05-22 00:55:48,895 | INFO | GPU: NVIDIA GB10
2026-05-22 00:55:49,029 | INFO | Hugging Face login succeeded
2026-05-22 00:55:49,029 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-22 00:55:49,029 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-22 00:55:49,029 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-22 00:55:49,803 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-22 00:56:57,725 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 00:56:59,883 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 00:57:06,291 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 00:57:07,271 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 00:57:07,625 | INFO | Train rows: 25335
2026-05-22 00:57:07,626 | INFO | Eval rows: 6334
2026-05-22 00:57:07,626 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>الـ سيرتيفيكاسيون في الـ لغات مهمة لـ خدمة في تونس؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>مهمة برشة للـ شركات الأجنبية والـ 'اوف-شور'. الـ (DELF) والـ (TOEIC) هوما الـ معيار اللي يقيموا بيه مستواك الحقيقي.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-22 00:57:07,626 | INFO | Detected checkpoint: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft/checkpoint-1584
2026-05-22 00:57:16,510 | INFO | Training started
2026-05-22 00:57:17,609 | INFO | Training finished
2026-05-22 00:57:17,609 | INFO | Saving model and tokenizer to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 00:57:18,156 | INFO | Saved training metrics to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft/training_metrics.json
2026-05-22 00:57:18,156 | INFO | Running preview generation on a Tunisian prompt
2026-05-22 00:57:19,396 | INFO | Preview prompt: عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟
2026-05-22 00:57:19,396 | INFO | Preview output: <|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|START_OF_TURN_TOKEN|><|USER_TOKEN|>عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>ح في في في في<|START_OF_TURN_TOKEN|>
2026-05-22 01:03:24,154 | INFO | Starting SFT fine-tuning job
2026-05-22 01:03:24,154 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 01:03:24,154 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 01:03:24,154 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 01:03:24,154 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 01:03:24,154 | INFO | Torch version: 2.11.0+cu130
2026-05-22 01:03:24,287 | INFO | CUDA available: True
2026-05-22 01:03:24,302 | INFO | GPU: NVIDIA GB10
2026-05-22 01:03:24,436 | INFO | Hugging Face login succeeded
2026-05-22 01:03:24,437 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-22 01:03:24,437 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-22 01:03:24,437 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-22 01:03:25,235 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-22 01:04:34,301 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 01:04:36,348 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 01:04:38,428 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 01:04:38,550 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 01:04:38,657 | INFO | Train rows: 25335
2026-05-22 01:04:38,658 | INFO | Eval rows: 6334
2026-05-22 01:04:38,658 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>الـ سيرتيفيكاسيون في الـ لغات مهمة لـ خدمة في تونس؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>مهمة برشة للـ شركات الأجنبية والـ 'اوف-شور'. الـ (DELF) والـ (TOEIC) هوما الـ معيار اللي يقيموا بيه مستواك الحقيقي.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-22 01:04:38,658 | INFO | force_restart enabled; checkpoint resume disabled
2026-05-22 01:04:40,699 | INFO | Training started
2026-05-22 12:34:27,241 | INFO | Starting SFT fine-tuning job
2026-05-22 12:34:27,242 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 12:34:27,242 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 12:34:27,242 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 12:34:27,242 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 12:34:27,242 | INFO | Torch version: 2.11.0+cu130
2026-05-22 12:34:27,374 | INFO | CUDA available: True
2026-05-22 12:34:27,399 | INFO | GPU: NVIDIA GB10
2026-05-22 12:34:27,562 | INFO | Hugging Face login succeeded
2026-05-22 12:34:27,563 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-22 12:34:27,563 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-22 12:34:27,563 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-22 12:34:28,414 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-22 12:35:55,107 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 12:35:57,709 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 12:35:59,906 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 12:36:00,063 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 12:36:00,200 | INFO | Train rows: 25335
2026-05-22 12:36:00,200 | INFO | Eval rows: 6334
2026-05-22 12:36:00,201 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>الـ سيرتيفيكاسيون في الـ لغات مهمة لـ خدمة في تونس؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>مهمة برشة للـ شركات الأجنبية والـ 'اوف-شور'. الـ (DELF) والـ (TOEIC) هوما الـ معيار اللي يقيموا بيه مستواك الحقيقي.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-22 12:36:00,201 | INFO | force_restart enabled; checkpoint resume disabled
2026-05-22 12:36:02,401 | INFO | Training started
2026-05-22 15:42:58,686 | INFO | Starting SFT fine-tuning job
2026-05-22 15:42:58,686 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 15:42:58,687 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 15:42:58,687 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 15:42:58,687 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 15:42:58,687 | INFO | Torch version: 2.11.0+cu130
2026-05-22 15:42:58,821 | INFO | CUDA available: True
2026-05-22 15:42:58,846 | INFO | GPU: NVIDIA GB10
2026-05-22 15:42:59,224 | INFO | Hugging Face login succeeded
2026-05-22 15:42:59,224 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-22 15:42:59,224 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-22 15:42:59,224 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-22 15:43:00,115 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-22 15:44:21,762 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 15:44:24,226 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 15:44:26,472 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 15:44:26,643 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 15:44:26,784 | INFO | Train rows: 25335
2026-05-22 15:44:26,784 | INFO | Eval rows: 6334
2026-05-22 15:44:26,786 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>الـ سيرتيفيكاسيون في الـ لغات مهمة لـ خدمة في تونس؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>مهمة برشة للـ شركات الأجنبية والـ 'اوف-شور'. الـ (DELF) والـ (TOEIC) هوما الـ معيار اللي يقيموا بيه مستواك الحقيقي.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-22 15:44:26,786 | INFO | force_restart enabled; checkpoint resume disabled
2026-05-22 15:44:29,343 | INFO | Training started
2026-05-22 15:44:50,452 | INFO | Starting SFT fine-tuning job
2026-05-22 15:44:50,452 | INFO | Output directory: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 15:44:50,452 | INFO | Base model: CohereLabs/aya-expanse-8b
2026-05-22 15:44:50,452 | INFO | CPT adapter: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 15:44:50,452 | INFO | Dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 15:44:50,452 | INFO | Torch version: 2.11.0+cu130
2026-05-22 15:44:50,586 | INFO | CUDA available: True
2026-05-22 15:44:50,602 | INFO | GPU: NVIDIA GB10
2026-05-22 15:44:50,763 | INFO | Hugging Face login succeeded
2026-05-22 15:44:50,764 | INFO | trl version 1.3.0: assistant_only_loss is fully supported
2026-05-22 15:44:50,764 | INFO | Loading configuration: batch_size=8, accumulation=4, warmup=0.03, weight_decay=0.0100
2026-05-22 15:44:50,764 | INFO | Loading tokenizer from base model: CohereLabs/aya-expanse-8b
2026-05-22 15:44:53,754 | INFO | Loading base model: CohereLabs/aya-expanse-8b with full bf16 precision (no quantization)
2026-05-22 15:46:13,999 | INFO | Loading CPT adapter from: /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-cpt-tunisian
2026-05-22 15:46:16,123 | INFO | Loading dataset: Syrinesmati/tunisian-question-response-dataset
2026-05-22 15:46:18,139 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 15:46:18,276 | INFO | Using dataset fields: question=instruction, answer=response
2026-05-22 15:46:18,398 | INFO | Train rows: 25335
2026-05-22 15:46:18,398 | INFO | Eval rows: 6334
2026-05-22 15:46:18,399 | INFO | Sample train example text: <BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>الـ سيرتيفيكاسيون في الـ لغات مهمة لـ خدمة في تونس؟<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>مهمة برشة للـ شركات الأجنبية والـ 'اوف-شور'. الـ (DELF) والـ (TOEIC) هوما الـ معيار اللي يقيموا بيه مستواك الحقيقي.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
2026-05-22 15:46:18,399 | INFO | force_restart enabled; checkpoint resume disabled
2026-05-22 15:46:20,378 | INFO | Training started
2026-05-22 22:06:47,054 | INFO | Training finished
2026-05-22 22:06:47,055 | INFO | Saving model and tokenizer to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft
2026-05-22 22:06:49,072 | INFO | Saved training metrics to /home/ala/TunisianDialogSystem/outputs/checkpoints/aya-expanse-8b-tunisian-sft/training_metrics.json
2026-05-22 22:06:49,072 | INFO | Running preview generation on a Tunisian prompt
2026-05-22 22:06:50,169 | INFO | Preview prompt: عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟
2026-05-22 22:06:50,169 | INFO | Preview output: <|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>أنت "التيجاني"، مساعد ذكاء اصطناعي تونسي 100%. جاوب بالتونسي الدارج فقط، وبالطول المناسب للسؤال: كان يلزم قصّر، وكان يلزم فسّر أكثر. ممنوع الهلوسة أو الخروج على الموضوع.<|START_OF_TURN_TOKEN|><|USER_TOKEN|>عسلامة، شنوة تنصحني نعمل كي نكون تعبان وبرشة؟<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>ح في في في<|START_OF_TURN_TOKEN|>