| 2026-06-22 13:09:34,105 [INFO] Seed: 42 |
| 2026-06-22 13:09:34,122 [INFO] Using CUDA: NVIDIA A100 80GB PCIe |
| 2026-06-22 13:09:34,122 [INFO] Enabled TF32 matmul/cudnn for CUDA |
| 2026-06-22 13:09:39,262 [INFO] LLM2Seq Model Summary |
| ================================================== |
| Encoder: McGill-NLP/LLM2Vec-Sheared-LLaMA-mntp |
| Trainable: True |
| LoRA: True |
| Adaptor: gated_residual |
| Layer Fusion: True |
| Fusion Type: tokenwise |
| Salience Gate: True |
| EncStack: True |
| Global Tokens: 16 |
| Decoder: |
| Layers: 8 |
| Hidden Size: 1024 |
| Heads: 16 |
| FFN Size: 4096 |
| ================================================== |
| Total params: 1,558,708,737 |
| Trainable params: 278,821,377 |
| Frozen params: 1,279,887,360 |
| 2026-06-22 13:09:39,959 [INFO] Enabled gradient checkpointing for encoder |
| 2026-06-22 13:09:40,214 [INFO] Resuming from checkpoint: runs/h200_llm2seq_phase1_warmup/best.pt |
| 2026-06-22 13:09:40,718 [INFO] Checkpoint load allowed 554 missing keys for stage=h200_phase2_lora_encoder context=resume, prefixes=('encoder.',) |
| 2026-06-22 13:09:40,719 [INFO] Reset global step because training.reset_global_step_on_resume=true |
| 2026-06-22 13:09:40,719 [INFO] Reset best eval loss because training.reset_best_eval_loss_on_resume=true |
| 2026-06-22 13:09:40,719 [INFO] Resumed at step 0 |
| 2026-06-22 13:09:40,780 [INFO] Hugging Face uploads enabled: repo=BienKieu/llm2seq-wikilingua path=checkpoints/h200_phase2_lora_encoder |
| 2026-06-22 13:09:40,780 [INFO] Train file: llm2seq_h200/data/processed/train.jsonl (13,999 examples) |
| 2026-06-22 13:09:40,781 [INFO] Eval file: llm2seq_h200/data/processed/validation.jsonl (1,680 examples) |
| 2026-06-22 13:09:40,781 [INFO] Starting training for 4 epochs (1,748 optimizer steps)... |
| 2026-06-22 13:09:40,781 [INFO] Batch size: 16 |
| 2026-06-22 13:09:40,781 [INFO] Gradient accumulation: 2 |
| 2026-06-22 13:09:40,781 [INFO] Effective batch size: 32 |
| 2026-06-22 13:09:40,781 [INFO] Steps per epoch: 437 |
| 2026-06-22 13:09:40,781 [INFO] Warmup steps: 52 |
| 2026-06-22 13:09:40,781 [INFO] Eval every: 437 optimizer steps |
| 2026-06-22 13:09:40,781 [INFO] Save every: 437 optimizer steps |
| 2026-06-22 13:09:40,781 [INFO] Mixed precision: fp16=False, bf16=True |
| 2026-06-22 13:09:41,003 [WARNING] `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`... |
| 2026-06-22 13:10:47,111 [INFO] Step 5/1748 | Loss: 1.5411 | CE: 1.5411 | LR: 4.81e-06 | Epoch: 1/4 |
| 2026-06-22 13:11:51,903 [INFO] Step 10/1748 | Loss: 1.4351 | CE: 1.4351 | LR: 9.62e-06 | Epoch: 1/4 |
| 2026-06-22 13:12:57,326 [INFO] Step 15/1748 | Loss: 1.3788 | CE: 1.3788 | LR: 1.44e-05 | Epoch: 1/4 |
| 2026-06-22 13:14:03,628 [INFO] Step 20/1748 | Loss: 1.3648 | CE: 1.3648 | LR: 1.92e-05 | Epoch: 1/4 |
| 2026-06-22 13:15:06,798 [INFO] Step 25/1748 | Loss: 1.2985 | CE: 1.2985 | LR: 2.40e-05 | Epoch: 1/4 |
| 2026-06-22 13:16:12,891 [INFO] Step 30/1748 | Loss: 1.2634 | CE: 1.2634 | LR: 2.88e-05 | Epoch: 1/4 |
| 2026-06-22 13:17:19,213 [INFO] Step 35/1748 | Loss: 1.2637 | CE: 1.2637 | LR: 3.37e-05 | Epoch: 1/4 |
| 2026-06-22 13:18:23,891 [INFO] Step 40/1748 | Loss: 1.1829 | CE: 1.1829 | LR: 3.85e-05 | Epoch: 1/4 |
| 2026-06-22 13:19:29,650 [INFO] Step 45/1748 | Loss: 1.2341 | CE: 1.2341 | LR: 4.33e-05 | Epoch: 1/4 |
| 2026-06-22 13:20:36,438 [INFO] Step 50/1748 | Loss: 1.1670 | CE: 1.1670 | LR: 4.81e-05 | Epoch: 1/4 |
| 2026-06-22 13:21:43,032 [INFO] Step 55/1748 | Loss: 1.1682 | CE: 1.1682 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:22:50,402 [INFO] Step 60/1748 | Loss: 1.1495 | CE: 1.1495 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:23:57,735 [INFO] Step 65/1748 | Loss: 1.1562 | CE: 1.1562 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:25:04,694 [INFO] Step 70/1748 | Loss: 1.1960 | CE: 1.1960 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:26:10,715 [INFO] Step 75/1748 | Loss: 1.1783 | CE: 1.1783 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:27:16,073 [INFO] Step 80/1748 | Loss: 1.1206 | CE: 1.1206 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:28:22,242 [INFO] Step 85/1748 | Loss: 1.1422 | CE: 1.1422 | LR: 5.00e-05 | Epoch: 1/4 |
| 2026-06-22 13:29:26,939 [INFO] Step 90/1748 | Loss: 1.1494 | CE: 1.1494 | LR: 4.99e-05 | Epoch: 1/4 |
| 2026-06-22 13:30:31,938 [INFO] Step 95/1748 | Loss: 1.0621 | CE: 1.0621 | LR: 4.99e-05 | Epoch: 1/4 |
| 2026-06-22 13:31:39,218 [INFO] Step 100/1748 | Loss: 1.1319 | CE: 1.1319 | LR: 4.99e-05 | Epoch: 1/4 |
| 2026-06-22 13:32:44,103 [INFO] Step 105/1748 | Loss: 1.1331 | CE: 1.1331 | LR: 4.99e-05 | Epoch: 1/4 |
| 2026-06-22 13:33:51,616 [INFO] Step 110/1748 | Loss: 1.1059 | CE: 1.1059 | LR: 4.99e-05 | Epoch: 1/4 |
| 2026-06-22 13:34:58,935 [INFO] Step 115/1748 | Loss: 1.1316 | CE: 1.1316 | LR: 4.98e-05 | Epoch: 1/4 |
| 2026-06-22 13:36:04,972 [INFO] Step 120/1748 | Loss: 1.1372 | CE: 1.1372 | LR: 4.98e-05 | Epoch: 1/4 |
| 2026-06-22 13:37:12,121 [INFO] Step 125/1748 | Loss: 1.1066 | CE: 1.1066 | LR: 4.98e-05 | Epoch: 1/4 |
| 2026-06-22 13:38:17,511 [INFO] Step 130/1748 | Loss: 1.1310 | CE: 1.1310 | LR: 4.97e-05 | Epoch: 1/4 |
| 2026-06-22 13:39:24,595 [INFO] Step 135/1748 | Loss: 1.1153 | CE: 1.1153 | LR: 4.97e-05 | Epoch: 1/4 |
| 2026-06-22 13:40:31,767 [INFO] Step 140/1748 | Loss: 1.0970 | CE: 1.0970 | LR: 4.97e-05 | Epoch: 1/4 |
| 2026-06-22 13:41:39,090 [INFO] Step 145/1748 | Loss: 1.0645 | CE: 1.0645 | LR: 4.96e-05 | Epoch: 1/4 |
| 2026-06-22 13:42:46,115 [INFO] Step 150/1748 | Loss: 1.0860 | CE: 1.0860 | LR: 4.96e-05 | Epoch: 1/4 |
| 2026-06-22 13:43:52,827 [INFO] Step 155/1748 | Loss: 1.0902 | CE: 1.0902 | LR: 4.95e-05 | Epoch: 1/4 |
| 2026-06-22 13:44:59,849 [INFO] Step 160/1748 | Loss: 1.1151 | CE: 1.1151 | LR: 4.95e-05 | Epoch: 1/4 |
| 2026-06-22 13:46:04,438 [INFO] Step 165/1748 | Loss: 1.1193 | CE: 1.1193 | LR: 4.95e-05 | Epoch: 1/4 |
| 2026-06-22 13:47:08,257 [INFO] Step 170/1748 | Loss: 1.0977 | CE: 1.0977 | LR: 4.94e-05 | Epoch: 1/4 |
| 2026-06-22 13:48:10,593 [INFO] Step 175/1748 | Loss: 1.1027 | CE: 1.1027 | LR: 4.94e-05 | Epoch: 1/4 |
| 2026-06-22 13:49:16,299 [INFO] Step 180/1748 | Loss: 1.0680 | CE: 1.0680 | LR: 4.93e-05 | Epoch: 1/4 |
| 2026-06-22 13:50:19,719 [INFO] Step 185/1748 | Loss: 1.0788 | CE: 1.0788 | LR: 4.92e-05 | Epoch: 1/4 |
| 2026-06-22 13:51:27,054 [INFO] Step 190/1748 | Loss: 1.1112 | CE: 1.1112 | LR: 4.92e-05 | Epoch: 1/4 |
| 2026-06-22 13:52:34,494 [INFO] Step 195/1748 | Loss: 1.0203 | CE: 1.0203 | LR: 4.91e-05 | Epoch: 1/4 |
| 2026-06-22 13:53:37,575 [INFO] Step 200/1748 | Loss: 1.0790 | CE: 1.0790 | LR: 4.91e-05 | Epoch: 1/4 |
| 2026-06-22 13:54:42,041 [INFO] Step 205/1748 | Loss: 1.1081 | CE: 1.1081 | LR: 4.90e-05 | Epoch: 1/4 |
| 2026-06-22 13:55:48,486 [INFO] Step 210/1748 | Loss: 1.1109 | CE: 1.1109 | LR: 4.89e-05 | Epoch: 1/4 |
| 2026-06-22 13:56:54,578 [INFO] Step 215/1748 | Loss: 1.1104 | CE: 1.1104 | LR: 4.89e-05 | Epoch: 1/4 |
| 2026-06-22 13:57:58,685 [INFO] Step 220/1748 | Loss: 1.1097 | CE: 1.1097 | LR: 4.88e-05 | Epoch: 1/4 |
| 2026-06-22 13:59:04,824 [INFO] Step 225/1748 | Loss: 1.1240 | CE: 1.1240 | LR: 4.87e-05 | Epoch: 1/4 |
| 2026-06-22 14:00:10,194 [INFO] Step 230/1748 | Loss: 1.1159 | CE: 1.1159 | LR: 4.87e-05 | Epoch: 1/4 |
| 2026-06-22 14:01:16,489 [INFO] Step 235/1748 | Loss: 1.1201 | CE: 1.1201 | LR: 4.86e-05 | Epoch: 1/4 |
| 2026-06-22 14:02:21,411 [INFO] Step 240/1748 | Loss: 1.0961 | CE: 1.0961 | LR: 4.85e-05 | Epoch: 1/4 |
| 2026-06-22 14:03:25,708 [INFO] Step 245/1748 | Loss: 1.0547 | CE: 1.0547 | LR: 4.84e-05 | Epoch: 1/4 |
| 2026-06-22 14:04:31,604 [INFO] Step 250/1748 | Loss: 1.1318 | CE: 1.1318 | LR: 4.83e-05 | Epoch: 1/4 |
| 2026-06-22 14:05:38,915 [INFO] Step 255/1748 | Loss: 1.0893 | CE: 1.0893 | LR: 4.83e-05 | Epoch: 1/4 |
| 2026-06-22 14:06:45,247 [INFO] Step 260/1748 | Loss: 1.0701 | CE: 1.0701 | LR: 4.82e-05 | Epoch: 1/4 |
| 2026-06-22 14:07:49,108 [INFO] Step 265/1748 | Loss: 1.0797 | CE: 1.0797 | LR: 4.81e-05 | Epoch: 1/4 |
| 2026-06-22 14:08:56,337 [INFO] Step 270/1748 | Loss: 1.0793 | CE: 1.0793 | LR: 4.80e-05 | Epoch: 1/4 |
| 2026-06-22 14:10:02,748 [INFO] Step 275/1748 | Loss: 1.0944 | CE: 1.0944 | LR: 4.79e-05 | Epoch: 1/4 |
| 2026-06-22 14:11:10,018 [INFO] Step 280/1748 | Loss: 1.1337 | CE: 1.1337 | LR: 4.78e-05 | Epoch: 1/4 |
| 2026-06-22 14:12:13,900 [INFO] Step 285/1748 | Loss: 1.0747 | CE: 1.0747 | LR: 4.77e-05 | Epoch: 1/4 |
| 2026-06-22 14:13:17,827 [INFO] Step 290/1748 | Loss: 1.1020 | CE: 1.1020 | LR: 4.76e-05 | Epoch: 1/4 |
| 2026-06-22 14:14:23,015 [INFO] Step 295/1748 | Loss: 1.1759 | CE: 1.1759 | LR: 4.75e-05 | Epoch: 1/4 |
| 2026-06-22 14:15:28,223 [INFO] Step 300/1748 | Loss: 1.1174 | CE: 1.1174 | LR: 4.74e-05 | Epoch: 1/4 |
| 2026-06-22 14:16:33,003 [INFO] Step 305/1748 | Loss: 1.0686 | CE: 1.0686 | LR: 4.73e-05 | Epoch: 1/4 |
| 2026-06-22 14:17:35,903 [INFO] Step 310/1748 | Loss: 1.1257 | CE: 1.1257 | LR: 4.72e-05 | Epoch: 1/4 |
| 2026-06-22 14:18:43,082 [INFO] Step 315/1748 | Loss: 1.1212 | CE: 1.1212 | LR: 4.71e-05 | Epoch: 1/4 |
| 2026-06-22 14:19:49,128 [INFO] Step 320/1748 | Loss: 1.0997 | CE: 1.0997 | LR: 4.70e-05 | Epoch: 1/4 |
| 2026-06-22 14:20:54,651 [INFO] Step 325/1748 | Loss: 1.1159 | CE: 1.1159 | LR: 4.69e-05 | Epoch: 1/4 |
| 2026-06-22 14:22:01,351 [INFO] Step 330/1748 | Loss: 1.0836 | CE: 1.0836 | LR: 4.68e-05 | Epoch: 1/4 |
| 2026-06-22 14:23:08,498 [INFO] Step 335/1748 | Loss: 1.1074 | CE: 1.1074 | LR: 4.66e-05 | Epoch: 1/4 |
| 2026-06-22 14:24:12,313 [INFO] Step 340/1748 | Loss: 1.1128 | CE: 1.1128 | LR: 4.65e-05 | Epoch: 1/4 |
| 2026-06-22 14:25:18,921 [INFO] Step 345/1748 | Loss: 1.0647 | CE: 1.0647 | LR: 4.64e-05 | Epoch: 1/4 |
| 2026-06-22 14:26:26,292 [INFO] Step 350/1748 | Loss: 1.0728 | CE: 1.0728 | LR: 4.63e-05 | Epoch: 1/4 |
| 2026-06-22 14:27:31,958 [INFO] Step 355/1748 | Loss: 1.1303 | CE: 1.1303 | LR: 4.62e-05 | Epoch: 1/4 |
| 2026-06-22 14:28:37,446 [INFO] Step 360/1748 | Loss: 1.1064 | CE: 1.1064 | LR: 4.60e-05 | Epoch: 1/4 |
| 2026-06-22 14:29:41,322 [INFO] Step 365/1748 | Loss: 1.1388 | CE: 1.1388 | LR: 4.59e-05 | Epoch: 1/4 |
| 2026-06-22 14:30:46,209 [INFO] Step 370/1748 | Loss: 1.1417 | CE: 1.1417 | LR: 4.58e-05 | Epoch: 1/4 |
| 2026-06-22 14:31:50,886 [INFO] Step 375/1748 | Loss: 1.1050 | CE: 1.1050 | LR: 4.57e-05 | Epoch: 1/4 |
| 2026-06-22 14:32:58,310 [INFO] Step 380/1748 | Loss: 1.1542 | CE: 1.1542 | LR: 4.55e-05 | Epoch: 1/4 |
| 2026-06-22 14:34:03,457 [INFO] Step 385/1748 | Loss: 1.1050 | CE: 1.1050 | LR: 4.54e-05 | Epoch: 1/4 |
| 2026-06-22 14:35:09,209 [INFO] Step 390/1748 | Loss: 1.0862 | CE: 1.0862 | LR: 4.53e-05 | Epoch: 1/4 |
| 2026-06-22 14:36:16,754 [INFO] Step 395/1748 | Loss: 1.0688 | CE: 1.0688 | LR: 4.51e-05 | Epoch: 1/4 |
| 2026-06-22 14:37:21,680 [INFO] Step 400/1748 | Loss: 1.0896 | CE: 1.0896 | LR: 4.50e-05 | Epoch: 1/4 |
| 2026-06-22 14:38:25,059 [INFO] Step 405/1748 | Loss: 1.0871 | CE: 1.0871 | LR: 4.48e-05 | Epoch: 1/4 |
| 2026-06-22 14:39:27,350 [INFO] Step 410/1748 | Loss: 1.1280 | CE: 1.1280 | LR: 4.47e-05 | Epoch: 1/4 |
| 2026-06-22 14:40:33,382 [INFO] Step 415/1748 | Loss: 1.1003 | CE: 1.1003 | LR: 4.46e-05 | Epoch: 1/4 |
| 2026-06-22 14:41:38,713 [INFO] Step 420/1748 | Loss: 1.0866 | CE: 1.0866 | LR: 4.44e-05 | Epoch: 1/4 |
| 2026-06-22 14:42:45,832 [INFO] Step 425/1748 | Loss: 1.0749 | CE: 1.0749 | LR: 4.43e-05 | Epoch: 1/4 |
| 2026-06-22 14:43:52,121 [INFO] Step 430/1748 | Loss: 1.0989 | CE: 1.0989 | LR: 4.41e-05 | Epoch: 1/4 |
| 2026-06-22 14:44:56,724 [INFO] Step 435/1748 | Loss: 1.0560 | CE: 1.0560 | LR: 4.40e-05 | Epoch: 1/4 |
| 2026-06-22 14:48:30,168 [INFO] Step 437 | Eval Loss: 1.3913 |
| 2026-06-22 14:48:31,206 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/best.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 14:48:31,792 [INFO] New best eval loss: 1.3913 |
| 2026-06-22 14:48:32,295 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/epoch_001.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 14:48:33,221 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/epoch_001.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/epochs/epoch_001_step_437.pt |
| 2026-06-22 14:49:02,861 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/config.yaml -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/config.yaml |
| 2026-06-22 14:49:03,337 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/train.log -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/train.log |
| 2026-06-22 14:49:03,804 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/checkpoint_manifest.json -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/checkpoint_manifest.json |
| 2026-06-22 14:49:04,628 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/best.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/best.pt |
| 2026-06-22 14:49:14,850 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/checkpoint_437.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 14:49:55,765 [INFO] Step 440/1748 | Loss: 1.0675 | CE: 1.0675 | LR: 4.38e-05 | Epoch: 2/4 |
| 2026-06-22 14:51:01,591 [INFO] Step 445/1748 | Loss: 1.0018 | CE: 1.0018 | LR: 4.37e-05 | Epoch: 2/4 |
| 2026-06-22 14:52:08,007 [INFO] Step 450/1748 | Loss: 0.9767 | CE: 0.9767 | LR: 4.35e-05 | Epoch: 2/4 |
| 2026-06-22 14:53:13,609 [INFO] Step 455/1748 | Loss: 1.0021 | CE: 1.0021 | LR: 4.34e-05 | Epoch: 2/4 |
| 2026-06-22 14:54:17,525 [INFO] Step 460/1748 | Loss: 0.9469 | CE: 0.9469 | LR: 4.32e-05 | Epoch: 2/4 |
| 2026-06-22 14:55:20,594 [INFO] Step 465/1748 | Loss: 0.9809 | CE: 0.9809 | LR: 4.30e-05 | Epoch: 2/4 |
| 2026-06-22 14:56:27,449 [INFO] Step 470/1748 | Loss: 0.9301 | CE: 0.9301 | LR: 4.29e-05 | Epoch: 2/4 |
| 2026-06-22 14:57:34,879 [INFO] Step 475/1748 | Loss: 1.0127 | CE: 1.0127 | LR: 4.27e-05 | Epoch: 2/4 |
| 2026-06-22 14:58:41,221 [INFO] Step 480/1748 | Loss: 0.9784 | CE: 0.9784 | LR: 4.25e-05 | Epoch: 2/4 |
| 2026-06-22 14:59:41,165 [INFO] Step 485/1748 | Loss: 0.9389 | CE: 0.9389 | LR: 4.24e-05 | Epoch: 2/4 |
| 2026-06-22 15:00:47,754 [INFO] Step 490/1748 | Loss: 0.9971 | CE: 0.9971 | LR: 4.22e-05 | Epoch: 2/4 |
| 2026-06-22 15:01:52,746 [INFO] Step 495/1748 | Loss: 1.0085 | CE: 1.0085 | LR: 4.20e-05 | Epoch: 2/4 |
| 2026-06-22 15:02:59,975 [INFO] Step 500/1748 | Loss: 1.0099 | CE: 1.0099 | LR: 4.19e-05 | Epoch: 2/4 |
| 2026-06-22 15:04:06,908 [INFO] Step 505/1748 | Loss: 1.0042 | CE: 1.0042 | LR: 4.17e-05 | Epoch: 2/4 |
| 2026-06-22 15:05:14,270 [INFO] Step 510/1748 | Loss: 0.9712 | CE: 0.9712 | LR: 4.15e-05 | Epoch: 2/4 |
| 2026-06-22 15:06:21,056 [INFO] Step 515/1748 | Loss: 0.9914 | CE: 0.9914 | LR: 4.14e-05 | Epoch: 2/4 |
| 2026-06-22 15:07:25,299 [INFO] Step 520/1748 | Loss: 1.0432 | CE: 1.0432 | LR: 4.12e-05 | Epoch: 2/4 |
| 2026-06-22 15:08:31,652 [INFO] Step 525/1748 | Loss: 0.9930 | CE: 0.9930 | LR: 4.10e-05 | Epoch: 2/4 |
| 2026-06-22 15:09:39,023 [INFO] Step 530/1748 | Loss: 0.9788 | CE: 0.9788 | LR: 4.08e-05 | Epoch: 2/4 |
| 2026-06-22 15:10:42,972 [INFO] Step 535/1748 | Loss: 0.9395 | CE: 0.9395 | LR: 4.06e-05 | Epoch: 2/4 |
| 2026-06-22 15:11:50,124 [INFO] Step 540/1748 | Loss: 0.9370 | CE: 0.9370 | LR: 4.05e-05 | Epoch: 2/4 |
| 2026-06-22 15:12:55,887 [INFO] Step 545/1748 | Loss: 0.9699 | CE: 0.9699 | LR: 4.03e-05 | Epoch: 2/4 |
| 2026-06-22 15:14:02,351 [INFO] Step 550/1748 | Loss: 0.9832 | CE: 0.9832 | LR: 4.01e-05 | Epoch: 2/4 |
| 2026-06-22 15:15:05,288 [INFO] Step 555/1748 | Loss: 0.9899 | CE: 0.9899 | LR: 3.99e-05 | Epoch: 2/4 |
| 2026-06-22 15:16:11,051 [INFO] Step 560/1748 | Loss: 1.0004 | CE: 1.0004 | LR: 3.97e-05 | Epoch: 2/4 |
| 2026-06-22 15:17:15,112 [INFO] Step 565/1748 | Loss: 1.0262 | CE: 1.0262 | LR: 3.95e-05 | Epoch: 2/4 |
| 2026-06-22 15:18:22,303 [INFO] Step 570/1748 | Loss: 0.9905 | CE: 0.9905 | LR: 3.93e-05 | Epoch: 2/4 |
| 2026-06-22 15:19:27,084 [INFO] Step 575/1748 | Loss: 0.9930 | CE: 0.9930 | LR: 3.92e-05 | Epoch: 2/4 |
| 2026-06-22 15:20:30,957 [INFO] Step 580/1748 | Loss: 1.0113 | CE: 1.0113 | LR: 3.90e-05 | Epoch: 2/4 |
| 2026-06-22 15:21:36,240 [INFO] Step 585/1748 | Loss: 1.0247 | CE: 1.0247 | LR: 3.88e-05 | Epoch: 2/4 |
| 2026-06-22 15:22:40,421 [INFO] Step 590/1748 | Loss: 0.9861 | CE: 0.9861 | LR: 3.86e-05 | Epoch: 2/4 |
| 2026-06-22 15:23:47,625 [INFO] Step 595/1748 | Loss: 1.0237 | CE: 1.0237 | LR: 3.84e-05 | Epoch: 2/4 |
| 2026-06-22 15:24:54,724 [INFO] Step 600/1748 | Loss: 0.9863 | CE: 0.9863 | LR: 3.82e-05 | Epoch: 2/4 |
| 2026-06-22 15:26:00,728 [INFO] Step 605/1748 | Loss: 1.0261 | CE: 1.0261 | LR: 3.80e-05 | Epoch: 2/4 |
| 2026-06-22 15:27:07,835 [INFO] Step 610/1748 | Loss: 0.9704 | CE: 0.9704 | LR: 3.78e-05 | Epoch: 2/4 |
| 2026-06-22 15:28:12,748 [INFO] Step 615/1748 | Loss: 1.0154 | CE: 1.0154 | LR: 3.76e-05 | Epoch: 2/4 |
| 2026-06-22 15:29:15,675 [INFO] Step 620/1748 | Loss: 0.9534 | CE: 0.9534 | LR: 3.74e-05 | Epoch: 2/4 |
| 2026-06-22 15:30:20,357 [INFO] Step 625/1748 | Loss: 1.0429 | CE: 1.0429 | LR: 3.72e-05 | Epoch: 2/4 |
| 2026-06-22 15:31:26,536 [INFO] Step 630/1748 | Loss: 0.9950 | CE: 0.9950 | LR: 3.70e-05 | Epoch: 2/4 |
| 2026-06-22 15:32:33,019 [INFO] Step 635/1748 | Loss: 0.9422 | CE: 0.9422 | LR: 3.68e-05 | Epoch: 2/4 |
| 2026-06-22 15:33:39,825 [INFO] Step 640/1748 | Loss: 1.0215 | CE: 1.0215 | LR: 3.66e-05 | Epoch: 2/4 |
| 2026-06-22 15:34:46,949 [INFO] Step 645/1748 | Loss: 1.0091 | CE: 1.0091 | LR: 3.64e-05 | Epoch: 2/4 |
| 2026-06-22 15:35:50,281 [INFO] Step 650/1748 | Loss: 0.9762 | CE: 0.9762 | LR: 3.62e-05 | Epoch: 2/4 |
| 2026-06-22 15:36:55,245 [INFO] Step 655/1748 | Loss: 0.9944 | CE: 0.9944 | LR: 3.60e-05 | Epoch: 2/4 |
| 2026-06-22 15:38:02,226 [INFO] Step 660/1748 | Loss: 1.0354 | CE: 1.0354 | LR: 3.58e-05 | Epoch: 2/4 |
| 2026-06-22 15:39:05,915 [INFO] Step 665/1748 | Loss: 0.9897 | CE: 0.9897 | LR: 3.55e-05 | Epoch: 2/4 |
| 2026-06-22 15:40:13,022 [INFO] Step 670/1748 | Loss: 0.9990 | CE: 0.9990 | LR: 3.53e-05 | Epoch: 2/4 |
| 2026-06-22 15:41:17,082 [INFO] Step 675/1748 | Loss: 0.9312 | CE: 0.9312 | LR: 3.51e-05 | Epoch: 2/4 |
| 2026-06-22 15:42:24,430 [INFO] Step 680/1748 | Loss: 0.9913 | CE: 0.9913 | LR: 3.49e-05 | Epoch: 2/4 |
| 2026-06-22 15:43:27,868 [INFO] Step 685/1748 | Loss: 1.0225 | CE: 1.0225 | LR: 3.47e-05 | Epoch: 2/4 |
| 2026-06-22 15:44:32,238 [INFO] Step 690/1748 | Loss: 0.9863 | CE: 0.9863 | LR: 3.45e-05 | Epoch: 2/4 |
| 2026-06-22 15:45:39,145 [INFO] Step 695/1748 | Loss: 0.9727 | CE: 0.9727 | LR: 3.43e-05 | Epoch: 2/4 |
| 2026-06-22 15:46:43,439 [INFO] Step 700/1748 | Loss: 0.9939 | CE: 0.9939 | LR: 3.41e-05 | Epoch: 2/4 |
| 2026-06-22 15:47:47,506 [INFO] Step 705/1748 | Loss: 1.0043 | CE: 1.0043 | LR: 3.38e-05 | Epoch: 2/4 |
| 2026-06-22 15:48:52,174 [INFO] Step 710/1748 | Loss: 0.9881 | CE: 0.9881 | LR: 3.36e-05 | Epoch: 2/4 |
| 2026-06-22 15:49:56,353 [INFO] Step 715/1748 | Loss: 0.9476 | CE: 0.9476 | LR: 3.34e-05 | Epoch: 2/4 |
| 2026-06-22 15:51:02,600 [INFO] Step 720/1748 | Loss: 0.9501 | CE: 0.9501 | LR: 3.32e-05 | Epoch: 2/4 |
| 2026-06-22 15:52:07,706 [INFO] Step 725/1748 | Loss: 1.0041 | CE: 1.0041 | LR: 3.30e-05 | Epoch: 2/4 |
| 2026-06-22 15:53:12,578 [INFO] Step 730/1748 | Loss: 0.9877 | CE: 0.9877 | LR: 3.27e-05 | Epoch: 2/4 |
| 2026-06-22 15:54:16,957 [INFO] Step 735/1748 | Loss: 0.9932 | CE: 0.9932 | LR: 3.25e-05 | Epoch: 2/4 |
| 2026-06-22 15:55:22,082 [INFO] Step 740/1748 | Loss: 1.0380 | CE: 1.0380 | LR: 3.23e-05 | Epoch: 2/4 |
| 2026-06-22 15:56:26,388 [INFO] Step 745/1748 | Loss: 0.9507 | CE: 0.9507 | LR: 3.21e-05 | Epoch: 2/4 |
| 2026-06-22 15:57:31,943 [INFO] Step 750/1748 | Loss: 0.9457 | CE: 0.9457 | LR: 3.19e-05 | Epoch: 2/4 |
| 2026-06-22 15:58:35,173 [INFO] Step 755/1748 | Loss: 1.0002 | CE: 1.0002 | LR: 3.16e-05 | Epoch: 2/4 |
| 2026-06-22 15:59:40,538 [INFO] Step 760/1748 | Loss: 1.0039 | CE: 1.0039 | LR: 3.14e-05 | Epoch: 2/4 |
| 2026-06-22 16:00:46,094 [INFO] Step 765/1748 | Loss: 0.9816 | CE: 0.9816 | LR: 3.12e-05 | Epoch: 2/4 |
| 2026-06-22 16:01:53,202 [INFO] Step 770/1748 | Loss: 1.0129 | CE: 1.0129 | LR: 3.10e-05 | Epoch: 2/4 |
| 2026-06-22 16:03:00,730 [INFO] Step 775/1748 | Loss: 0.9791 | CE: 0.9791 | LR: 3.07e-05 | Epoch: 2/4 |
| 2026-06-22 16:04:06,398 [INFO] Step 780/1748 | Loss: 0.9903 | CE: 0.9903 | LR: 3.05e-05 | Epoch: 2/4 |
| 2026-06-22 16:05:08,992 [INFO] Step 785/1748 | Loss: 0.9772 | CE: 0.9772 | LR: 3.03e-05 | Epoch: 2/4 |
| 2026-06-22 16:06:15,066 [INFO] Step 790/1748 | Loss: 0.9427 | CE: 0.9427 | LR: 3.01e-05 | Epoch: 2/4 |
| 2026-06-22 16:07:22,443 [INFO] Step 795/1748 | Loss: 0.9744 | CE: 0.9744 | LR: 2.98e-05 | Epoch: 2/4 |
| 2026-06-22 16:08:23,922 [INFO] Step 800/1748 | Loss: 1.0205 | CE: 1.0205 | LR: 2.96e-05 | Epoch: 2/4 |
| 2026-06-22 16:09:29,776 [INFO] Step 805/1748 | Loss: 0.9967 | CE: 0.9967 | LR: 2.94e-05 | Epoch: 2/4 |
| 2026-06-22 16:10:36,567 [INFO] Step 810/1748 | Loss: 0.9569 | CE: 0.9569 | LR: 2.91e-05 | Epoch: 2/4 |
| 2026-06-22 16:11:41,467 [INFO] Step 815/1748 | Loss: 0.9460 | CE: 0.9460 | LR: 2.89e-05 | Epoch: 2/4 |
| 2026-06-22 16:12:47,036 [INFO] Step 820/1748 | Loss: 1.0088 | CE: 1.0088 | LR: 2.87e-05 | Epoch: 2/4 |
| 2026-06-22 16:13:54,507 [INFO] Step 825/1748 | Loss: 0.9622 | CE: 0.9622 | LR: 2.85e-05 | Epoch: 2/4 |
| 2026-06-22 16:14:55,264 [INFO] Step 830/1748 | Loss: 1.0036 | CE: 1.0036 | LR: 2.82e-05 | Epoch: 2/4 |
| 2026-06-22 16:16:01,186 [INFO] Step 835/1748 | Loss: 0.9869 | CE: 0.9869 | LR: 2.80e-05 | Epoch: 2/4 |
| 2026-06-22 16:17:06,521 [INFO] Step 840/1748 | Loss: 0.9625 | CE: 0.9625 | LR: 2.78e-05 | Epoch: 2/4 |
| 2026-06-22 16:18:09,766 [INFO] Step 845/1748 | Loss: 0.9860 | CE: 0.9860 | LR: 2.75e-05 | Epoch: 2/4 |
| 2026-06-22 16:19:14,188 [INFO] Step 850/1748 | Loss: 0.9615 | CE: 0.9615 | LR: 2.73e-05 | Epoch: 2/4 |
| 2026-06-22 16:20:20,242 [INFO] Step 855/1748 | Loss: 0.9897 | CE: 0.9897 | LR: 2.71e-05 | Epoch: 2/4 |
| 2026-06-22 16:21:26,333 [INFO] Step 860/1748 | Loss: 0.9635 | CE: 0.9635 | LR: 2.69e-05 | Epoch: 2/4 |
| 2026-06-22 16:22:31,928 [INFO] Step 865/1748 | Loss: 1.0036 | CE: 1.0036 | LR: 2.66e-05 | Epoch: 2/4 |
| 2026-06-22 16:23:36,930 [INFO] Step 870/1748 | Loss: 0.9904 | CE: 0.9904 | LR: 2.64e-05 | Epoch: 2/4 |
| 2026-06-22 16:27:36,082 [INFO] Step 874 | Eval Loss: 1.3895 |
| 2026-06-22 16:27:37,530 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/best.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 16:27:38,144 [INFO] New best eval loss: 1.3895 |
| 2026-06-22 16:42:46,507 [INFO] Seed: 42 |
| 2026-06-22 16:42:46,524 [INFO] Using CUDA: NVIDIA A100 80GB PCIe |
| 2026-06-22 16:42:46,524 [INFO] Enabled TF32 matmul/cudnn for CUDA |
| 2026-06-22 16:42:51,618 [INFO] LLM2Seq Model Summary |
| ================================================== |
| Encoder: McGill-NLP/LLM2Vec-Sheared-LLaMA-mntp |
| Trainable: True |
| LoRA: True |
| Adaptor: gated_residual |
| Layer Fusion: True |
| Fusion Type: tokenwise |
| Salience Gate: True |
| EncStack: True |
| Global Tokens: 16 |
| Decoder: |
| Layers: 8 |
| Hidden Size: 1024 |
| Heads: 16 |
| FFN Size: 4096 |
| ================================================== |
| Total params: 1,558,708,737 |
| Trainable params: 278,821,377 |
| Frozen params: 1,279,887,360 |
| 2026-06-22 16:42:52,209 [INFO] Enabled gradient checkpointing for encoder |
| 2026-06-22 16:42:52,461 [INFO] Resuming from checkpoint: runs/h200_llm2seq_phase2_lora_encoder/best.pt |
| 2026-06-22 16:42:52,959 [INFO] Checkpoint load allowed 218 missing keys for stage=h200_phase2_lora_encoder context=resume, prefixes=('encoder.',) |
| 2026-06-22 16:42:52,959 [INFO] Reset global step because training.reset_global_step_on_resume=true |
| 2026-06-22 16:42:52,959 [INFO] Reset best eval loss because training.reset_best_eval_loss_on_resume=true |
| 2026-06-22 16:42:52,959 [INFO] Resumed at step 0 |
| 2026-06-22 16:42:53,020 [INFO] Hugging Face uploads enabled: repo=BienKieu/llm2seq-wikilingua path=checkpoints/h200_phase2_lora_encoder |
| 2026-06-22 16:42:53,020 [INFO] Train file: llm2seq_h200/data/processed/train.jsonl (13,999 examples) |
| 2026-06-22 16:42:53,020 [INFO] Eval file: llm2seq_h200/data/processed/validation.jsonl (1,680 examples) |
| 2026-06-22 16:42:53,020 [INFO] Starting training for 2 epochs (874 optimizer steps)... |
| 2026-06-22 16:42:53,020 [INFO] Batch size: 16 |
| 2026-06-22 16:42:53,020 [INFO] Gradient accumulation: 2 |
| 2026-06-22 16:42:53,020 [INFO] Effective batch size: 32 |
| 2026-06-22 16:42:53,020 [INFO] Steps per epoch: 437 |
| 2026-06-22 16:42:53,020 [INFO] Warmup steps: 26 |
| 2026-06-22 16:42:53,020 [INFO] Eval every: 437 optimizer steps |
| 2026-06-22 16:42:53,020 [INFO] Save every: 437 optimizer steps |
| 2026-06-22 16:42:53,020 [INFO] Mixed precision: fp16=False, bf16=True |
| 2026-06-22 16:42:53,290 [WARNING] `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`... |
| 2026-06-22 16:43:59,651 [INFO] Step 5/874 | Loss: 0.9483 | CE: 0.9483 | LR: 9.62e-06 | Epoch: 1/2 |
| 2026-06-22 16:45:04,618 [INFO] Step 10/874 | Loss: 0.9112 | CE: 0.9112 | LR: 1.92e-05 | Epoch: 1/2 |
| 2026-06-22 16:46:10,189 [INFO] Step 15/874 | Loss: 0.9204 | CE: 0.9204 | LR: 2.88e-05 | Epoch: 1/2 |
| 2026-06-22 16:47:16,629 [INFO] Step 20/874 | Loss: 0.9214 | CE: 0.9214 | LR: 3.85e-05 | Epoch: 1/2 |
| 2026-06-22 16:48:19,942 [INFO] Step 25/874 | Loss: 0.9024 | CE: 0.9024 | LR: 4.81e-05 | Epoch: 1/2 |
| 2026-06-22 16:49:26,165 [INFO] Step 30/874 | Loss: 0.8728 | CE: 0.8728 | LR: 5.00e-05 | Epoch: 1/2 |
| 2026-06-22 16:50:32,691 [INFO] Step 35/874 | Loss: 0.8813 | CE: 0.8813 | LR: 5.00e-05 | Epoch: 1/2 |
| 2026-06-22 16:51:37,525 [INFO] Step 40/874 | Loss: 0.8341 | CE: 0.8341 | LR: 5.00e-05 | Epoch: 1/2 |
| 2026-06-22 16:52:43,468 [INFO] Step 45/874 | Loss: 0.8893 | CE: 0.8893 | LR: 4.99e-05 | Epoch: 1/2 |
| 2026-06-22 16:53:50,378 [INFO] Step 50/874 | Loss: 0.8299 | CE: 0.8299 | LR: 4.99e-05 | Epoch: 1/2 |
| 2026-06-22 16:54:57,058 [INFO] Step 55/874 | Loss: 0.8488 | CE: 0.8488 | LR: 4.99e-05 | Epoch: 1/2 |
| 2026-06-22 16:56:04,521 [INFO] Step 60/874 | Loss: 0.8315 | CE: 0.8315 | LR: 4.98e-05 | Epoch: 1/2 |
| 2026-06-22 16:57:11,911 [INFO] Step 65/874 | Loss: 0.8333 | CE: 0.8333 | LR: 4.97e-05 | Epoch: 1/2 |
| 2026-06-22 16:58:18,984 [INFO] Step 70/874 | Loss: 0.8728 | CE: 0.8728 | LR: 4.97e-05 | Epoch: 1/2 |
| 2026-06-22 16:59:25,078 [INFO] Step 75/874 | Loss: 0.8462 | CE: 0.8462 | LR: 4.96e-05 | Epoch: 1/2 |
| 2026-06-22 17:00:30,527 [INFO] Step 80/874 | Loss: 0.8005 | CE: 0.8005 | LR: 4.95e-05 | Epoch: 1/2 |
| 2026-06-22 17:01:36,863 [INFO] Step 85/874 | Loss: 0.8164 | CE: 0.8164 | LR: 4.94e-05 | Epoch: 1/2 |
| 2026-06-22 17:02:41,689 [INFO] Step 90/874 | Loss: 0.8239 | CE: 0.8239 | LR: 4.93e-05 | Epoch: 1/2 |
| 2026-06-22 17:03:46,756 [INFO] Step 95/874 | Loss: 0.7703 | CE: 0.7703 | LR: 4.92e-05 | Epoch: 1/2 |
| 2026-06-22 17:04:54,070 [INFO] Step 100/874 | Loss: 0.8371 | CE: 0.8371 | LR: 4.91e-05 | Epoch: 1/2 |
| 2026-06-22 17:05:59,032 [INFO] Step 105/874 | Loss: 0.8346 | CE: 0.8346 | LR: 4.89e-05 | Epoch: 1/2 |
| 2026-06-22 17:07:06,645 [INFO] Step 110/874 | Loss: 0.8172 | CE: 0.8172 | LR: 4.88e-05 | Epoch: 1/2 |
| 2026-06-22 17:08:13,897 [INFO] Step 115/874 | Loss: 0.8265 | CE: 0.8265 | LR: 4.87e-05 | Epoch: 1/2 |
| 2026-06-22 17:09:20,010 [INFO] Step 120/874 | Loss: 0.8314 | CE: 0.8314 | LR: 4.85e-05 | Epoch: 1/2 |
| 2026-06-22 17:10:27,258 [INFO] Step 125/874 | Loss: 0.8111 | CE: 0.8111 | LR: 4.83e-05 | Epoch: 1/2 |
| 2026-06-22 17:11:32,659 [INFO] Step 130/874 | Loss: 0.8198 | CE: 0.8198 | LR: 4.82e-05 | Epoch: 1/2 |
| 2026-06-22 17:12:39,780 [INFO] Step 135/874 | Loss: 0.8046 | CE: 0.8046 | LR: 4.80e-05 | Epoch: 1/2 |
| 2026-06-22 17:13:46,973 [INFO] Step 140/874 | Loss: 0.7905 | CE: 0.7905 | LR: 4.78e-05 | Epoch: 1/2 |
| 2026-06-22 17:14:54,376 [INFO] Step 145/874 | Loss: 0.7689 | CE: 0.7689 | LR: 4.76e-05 | Epoch: 1/2 |
| 2026-06-22 17:16:01,471 [INFO] Step 150/874 | Loss: 0.7868 | CE: 0.7868 | LR: 4.74e-05 | Epoch: 1/2 |
| 2026-06-22 17:17:08,325 [INFO] Step 155/874 | Loss: 0.7957 | CE: 0.7957 | LR: 4.72e-05 | Epoch: 1/2 |
| 2026-06-22 17:18:15,360 [INFO] Step 160/874 | Loss: 0.8039 | CE: 0.8039 | LR: 4.70e-05 | Epoch: 1/2 |
| 2026-06-22 17:19:19,921 [INFO] Step 165/874 | Loss: 0.8169 | CE: 0.8169 | LR: 4.68e-05 | Epoch: 1/2 |
| 2026-06-22 17:20:23,750 [INFO] Step 170/874 | Loss: 0.8050 | CE: 0.8050 | LR: 4.65e-05 | Epoch: 1/2 |
| 2026-06-22 17:21:26,134 [INFO] Step 175/874 | Loss: 0.8061 | CE: 0.8061 | LR: 4.63e-05 | Epoch: 1/2 |
| 2026-06-22 17:22:31,893 [INFO] Step 180/874 | Loss: 0.7662 | CE: 0.7662 | LR: 4.60e-05 | Epoch: 1/2 |
| 2026-06-22 17:23:35,409 [INFO] Step 185/874 | Loss: 0.7755 | CE: 0.7755 | LR: 4.58e-05 | Epoch: 1/2 |
| 2026-06-22 17:24:42,849 [INFO] Step 190/874 | Loss: 0.8107 | CE: 0.8107 | LR: 4.55e-05 | Epoch: 1/2 |
| 2026-06-22 17:25:50,252 [INFO] Step 195/874 | Loss: 0.7431 | CE: 0.7431 | LR: 4.53e-05 | Epoch: 1/2 |
| 2026-06-22 17:26:53,381 [INFO] Step 200/874 | Loss: 0.7695 | CE: 0.7695 | LR: 4.50e-05 | Epoch: 1/2 |
| 2026-06-22 17:27:57,929 [INFO] Step 205/874 | Loss: 0.7999 | CE: 0.7999 | LR: 4.47e-05 | Epoch: 1/2 |
| 2026-06-22 17:29:04,399 [INFO] Step 210/874 | Loss: 0.8081 | CE: 0.8081 | LR: 4.44e-05 | Epoch: 1/2 |
| 2026-06-22 17:30:10,561 [INFO] Step 215/874 | Loss: 0.8112 | CE: 0.8112 | LR: 4.41e-05 | Epoch: 1/2 |
| 2026-06-22 17:31:14,734 [INFO] Step 220/874 | Loss: 0.8124 | CE: 0.8124 | LR: 4.38e-05 | Epoch: 1/2 |
| 2026-06-22 17:32:20,952 [INFO] Step 225/874 | Loss: 0.8257 | CE: 0.8257 | LR: 4.35e-05 | Epoch: 1/2 |
| 2026-06-22 17:33:26,379 [INFO] Step 230/874 | Loss: 0.8138 | CE: 0.8138 | LR: 4.32e-05 | Epoch: 1/2 |
| 2026-06-22 17:34:32,666 [INFO] Step 235/874 | Loss: 0.8117 | CE: 0.8117 | LR: 4.29e-05 | Epoch: 1/2 |
| 2026-06-22 17:35:37,581 [INFO] Step 240/874 | Loss: 0.7866 | CE: 0.7866 | LR: 4.25e-05 | Epoch: 1/2 |
| 2026-06-22 17:36:41,948 [INFO] Step 245/874 | Loss: 0.7625 | CE: 0.7625 | LR: 4.22e-05 | Epoch: 1/2 |
| 2026-06-22 17:37:47,922 [INFO] Step 250/874 | Loss: 0.8282 | CE: 0.8282 | LR: 4.19e-05 | Epoch: 1/2 |
| 2026-06-22 17:38:55,291 [INFO] Step 255/874 | Loss: 0.7863 | CE: 0.7863 | LR: 4.15e-05 | Epoch: 1/2 |
| 2026-06-22 17:40:01,650 [INFO] Step 260/874 | Loss: 0.7724 | CE: 0.7724 | LR: 4.12e-05 | Epoch: 1/2 |
| 2026-06-22 17:41:05,591 [INFO] Step 265/874 | Loss: 0.7789 | CE: 0.7789 | LR: 4.08e-05 | Epoch: 1/2 |
| 2026-06-22 17:42:12,833 [INFO] Step 270/874 | Loss: 0.7707 | CE: 0.7707 | LR: 4.05e-05 | Epoch: 1/2 |
| 2026-06-22 17:43:19,238 [INFO] Step 275/874 | Loss: 0.7959 | CE: 0.7959 | LR: 4.01e-05 | Epoch: 1/2 |
| 2026-06-22 17:44:26,519 [INFO] Step 280/874 | Loss: 0.8279 | CE: 0.8279 | LR: 3.97e-05 | Epoch: 1/2 |
| 2026-06-22 17:45:30,494 [INFO] Step 285/874 | Loss: 0.7839 | CE: 0.7839 | LR: 3.93e-05 | Epoch: 1/2 |
| 2026-06-22 17:46:34,491 [INFO] Step 290/874 | Loss: 0.7998 | CE: 0.7998 | LR: 3.90e-05 | Epoch: 1/2 |
| 2026-06-22 17:47:39,709 [INFO] Step 295/874 | Loss: 0.8587 | CE: 0.8587 | LR: 3.86e-05 | Epoch: 1/2 |
| 2026-06-22 17:48:45,015 [INFO] Step 300/874 | Loss: 0.8047 | CE: 0.8047 | LR: 3.82e-05 | Epoch: 1/2 |
| 2026-06-22 17:49:49,819 [INFO] Step 305/874 | Loss: 0.7720 | CE: 0.7720 | LR: 3.78e-05 | Epoch: 1/2 |
| 2026-06-22 17:50:52,737 [INFO] Step 310/874 | Loss: 0.8027 | CE: 0.8027 | LR: 3.74e-05 | Epoch: 1/2 |
| 2026-06-22 17:51:59,884 [INFO] Step 315/874 | Loss: 0.8179 | CE: 0.8179 | LR: 3.70e-05 | Epoch: 1/2 |
| 2026-06-22 17:53:05,852 [INFO] Step 320/874 | Loss: 0.7906 | CE: 0.7906 | LR: 3.66e-05 | Epoch: 1/2 |
| 2026-06-22 17:54:11,353 [INFO] Step 325/874 | Loss: 0.7973 | CE: 0.7973 | LR: 3.62e-05 | Epoch: 1/2 |
| 2026-06-22 17:55:18,044 [INFO] Step 330/874 | Loss: 0.7764 | CE: 0.7764 | LR: 3.58e-05 | Epoch: 1/2 |
| 2026-06-22 17:56:25,219 [INFO] Step 335/874 | Loss: 0.7987 | CE: 0.7987 | LR: 3.53e-05 | Epoch: 1/2 |
| 2026-06-22 17:57:29,023 [INFO] Step 340/874 | Loss: 0.8041 | CE: 0.8041 | LR: 3.49e-05 | Epoch: 1/2 |
| 2026-06-22 17:58:35,639 [INFO] Step 345/874 | Loss: 0.7577 | CE: 0.7577 | LR: 3.45e-05 | Epoch: 1/2 |
| 2026-06-22 17:59:42,933 [INFO] Step 350/874 | Loss: 0.7675 | CE: 0.7675 | LR: 3.41e-05 | Epoch: 1/2 |
| 2026-06-22 18:00:48,601 [INFO] Step 355/874 | Loss: 0.8113 | CE: 0.8113 | LR: 3.36e-05 | Epoch: 1/2 |
| 2026-06-22 18:01:54,273 [INFO] Step 360/874 | Loss: 0.7949 | CE: 0.7949 | LR: 3.32e-05 | Epoch: 1/2 |
| 2026-06-22 18:02:58,178 [INFO] Step 365/874 | Loss: 0.8143 | CE: 0.8143 | LR: 3.27e-05 | Epoch: 1/2 |
| 2026-06-22 18:04:03,078 [INFO] Step 370/874 | Loss: 0.8211 | CE: 0.8211 | LR: 3.23e-05 | Epoch: 1/2 |
| 2026-06-22 18:05:07,526 [INFO] Step 375/874 | Loss: 0.7993 | CE: 0.7993 | LR: 3.19e-05 | Epoch: 1/2 |
| 2026-06-22 18:06:14,943 [INFO] Step 380/874 | Loss: 0.8504 | CE: 0.8504 | LR: 3.14e-05 | Epoch: 1/2 |
| 2026-06-22 18:07:20,072 [INFO] Step 385/874 | Loss: 0.7840 | CE: 0.7840 | LR: 3.10e-05 | Epoch: 1/2 |
| 2026-06-22 18:08:25,826 [INFO] Step 390/874 | Loss: 0.7754 | CE: 0.7754 | LR: 3.05e-05 | Epoch: 1/2 |
| 2026-06-22 18:09:33,391 [INFO] Step 395/874 | Loss: 0.7756 | CE: 0.7756 | LR: 3.01e-05 | Epoch: 1/2 |
| 2026-06-22 18:10:38,296 [INFO] Step 400/874 | Loss: 0.7913 | CE: 0.7913 | LR: 2.96e-05 | Epoch: 1/2 |
| 2026-06-22 18:11:41,717 [INFO] Step 405/874 | Loss: 0.7742 | CE: 0.7742 | LR: 2.91e-05 | Epoch: 1/2 |
| 2026-06-22 18:12:44,030 [INFO] Step 410/874 | Loss: 0.8034 | CE: 0.8034 | LR: 2.87e-05 | Epoch: 1/2 |
| 2026-06-22 18:13:50,078 [INFO] Step 415/874 | Loss: 0.7887 | CE: 0.7887 | LR: 2.82e-05 | Epoch: 1/2 |
| 2026-06-22 18:14:55,445 [INFO] Step 420/874 | Loss: 0.7852 | CE: 0.7852 | LR: 2.78e-05 | Epoch: 1/2 |
| 2026-06-22 18:16:02,579 [INFO] Step 425/874 | Loss: 0.7805 | CE: 0.7805 | LR: 2.73e-05 | Epoch: 1/2 |
| 2026-06-22 18:17:08,912 [INFO] Step 430/874 | Loss: 0.8019 | CE: 0.8019 | LR: 2.69e-05 | Epoch: 1/2 |
| 2026-06-22 18:18:13,675 [INFO] Step 435/874 | Loss: 0.7519 | CE: 0.7519 | LR: 2.64e-05 | Epoch: 1/2 |
| 2026-06-22 18:21:47,083 [INFO] Step 437 | Eval Loss: 1.4775 |
| 2026-06-22 18:21:48,374 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/best.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 18:21:48,982 [INFO] New best eval loss: 1.4775 |
| 2026-06-22 18:21:49,536 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/epoch_001.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 18:21:50,337 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/epoch_001.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/epochs/epoch_001_step_437.pt |
| 2026-06-22 18:22:13,383 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/config.yaml -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/config.yaml |
| 2026-06-22 18:22:13,705 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/train.log -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/train.log |
| 2026-06-22 18:22:14,071 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/checkpoint_manifest.json -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/checkpoint_manifest.json |
| 2026-06-22 18:22:14,144 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 18:22:14,772 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/best.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/best.pt |
| 2026-06-22 18:22:25,324 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/checkpoint_437.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 18:23:06,031 [INFO] Step 440/874 | Loss: 0.7526 | CE: 0.7526 | LR: 2.59e-05 | Epoch: 2/2 |
| 2026-06-22 18:24:11,928 [INFO] Step 445/874 | Loss: 0.6760 | CE: 0.6760 | LR: 2.55e-05 | Epoch: 2/2 |
| 2026-06-22 18:25:18,237 [INFO] Step 450/874 | Loss: 0.6585 | CE: 0.6585 | LR: 2.50e-05 | Epoch: 2/2 |
| 2026-06-22 18:26:23,672 [INFO] Step 455/874 | Loss: 0.6663 | CE: 0.6663 | LR: 2.45e-05 | Epoch: 2/2 |
| 2026-06-22 18:27:27,604 [INFO] Step 460/874 | Loss: 0.6314 | CE: 0.6314 | LR: 2.41e-05 | Epoch: 2/2 |
| 2026-06-22 18:28:30,667 [INFO] Step 465/874 | Loss: 0.6577 | CE: 0.6577 | LR: 2.36e-05 | Epoch: 2/2 |
| 2026-06-22 18:29:37,585 [INFO] Step 470/874 | Loss: 0.6223 | CE: 0.6223 | LR: 2.31e-05 | Epoch: 2/2 |
| 2026-06-22 18:30:45,121 [INFO] Step 475/874 | Loss: 0.6784 | CE: 0.6784 | LR: 2.27e-05 | Epoch: 2/2 |
| 2026-06-22 18:31:51,490 [INFO] Step 480/874 | Loss: 0.6582 | CE: 0.6582 | LR: 2.22e-05 | Epoch: 2/2 |
| 2026-06-22 18:32:51,515 [INFO] Step 485/874 | Loss: 0.6192 | CE: 0.6192 | LR: 2.18e-05 | Epoch: 2/2 |
| 2026-06-22 18:33:58,174 [INFO] Step 490/874 | Loss: 0.6675 | CE: 0.6675 | LR: 2.13e-05 | Epoch: 2/2 |
| 2026-06-22 18:35:03,199 [INFO] Step 495/874 | Loss: 0.6733 | CE: 0.6733 | LR: 2.09e-05 | Epoch: 2/2 |
| 2026-06-22 18:36:10,421 [INFO] Step 500/874 | Loss: 0.6779 | CE: 0.6779 | LR: 2.04e-05 | Epoch: 2/2 |
| 2026-06-22 18:37:17,323 [INFO] Step 505/874 | Loss: 0.6684 | CE: 0.6684 | LR: 1.99e-05 | Epoch: 2/2 |
| 2026-06-22 18:38:24,713 [INFO] Step 510/874 | Loss: 0.6440 | CE: 0.6440 | LR: 1.95e-05 | Epoch: 2/2 |
| 2026-06-22 18:39:31,511 [INFO] Step 515/874 | Loss: 0.6573 | CE: 0.6573 | LR: 1.90e-05 | Epoch: 2/2 |
| 2026-06-22 18:40:35,820 [INFO] Step 520/874 | Loss: 0.7002 | CE: 0.7002 | LR: 1.86e-05 | Epoch: 2/2 |
| 2026-06-22 18:41:42,196 [INFO] Step 525/874 | Loss: 0.6571 | CE: 0.6571 | LR: 1.81e-05 | Epoch: 2/2 |
| 2026-06-22 18:42:49,510 [INFO] Step 530/874 | Loss: 0.6473 | CE: 0.6473 | LR: 1.77e-05 | Epoch: 2/2 |
| 2026-06-22 18:43:53,557 [INFO] Step 535/874 | Loss: 0.6248 | CE: 0.6248 | LR: 1.73e-05 | Epoch: 2/2 |
| 2026-06-22 18:45:00,697 [INFO] Step 540/874 | Loss: 0.6269 | CE: 0.6269 | LR: 1.68e-05 | Epoch: 2/2 |
| 2026-06-22 18:46:06,469 [INFO] Step 545/874 | Loss: 0.6463 | CE: 0.6463 | LR: 1.64e-05 | Epoch: 2/2 |
| 2026-06-22 18:47:12,936 [INFO] Step 550/874 | Loss: 0.6561 | CE: 0.6561 | LR: 1.59e-05 | Epoch: 2/2 |
| 2026-06-22 18:48:15,872 [INFO] Step 555/874 | Loss: 0.6494 | CE: 0.6494 | LR: 1.55e-05 | Epoch: 2/2 |
| 2026-06-22 18:49:21,795 [INFO] Step 560/874 | Loss: 0.6783 | CE: 0.6783 | LR: 1.51e-05 | Epoch: 2/2 |
| 2026-06-22 18:50:26,199 [INFO] Step 565/874 | Loss: 0.6812 | CE: 0.6812 | LR: 1.47e-05 | Epoch: 2/2 |
| 2026-06-22 18:51:33,721 [INFO] Step 570/874 | Loss: 0.6749 | CE: 0.6749 | LR: 1.42e-05 | Epoch: 2/2 |
| 2026-06-22 18:52:38,482 [INFO] Step 575/874 | Loss: 0.6687 | CE: 0.6687 | LR: 1.38e-05 | Epoch: 2/2 |
| 2026-06-22 18:53:42,361 [INFO] Step 580/874 | Loss: 0.6773 | CE: 0.6773 | LR: 1.34e-05 | Epoch: 2/2 |
| 2026-06-22 18:54:47,657 [INFO] Step 585/874 | Loss: 0.6899 | CE: 0.6899 | LR: 1.30e-05 | Epoch: 2/2 |
| 2026-06-22 18:55:52,042 [INFO] Step 590/874 | Loss: 0.6634 | CE: 0.6634 | LR: 1.26e-05 | Epoch: 2/2 |
| 2026-06-22 18:56:59,644 [INFO] Step 595/874 | Loss: 0.6923 | CE: 0.6923 | LR: 1.22e-05 | Epoch: 2/2 |
| 2026-06-22 18:58:07,066 [INFO] Step 600/874 | Loss: 0.6613 | CE: 0.6613 | LR: 1.18e-05 | Epoch: 2/2 |
| 2026-06-22 18:59:13,375 [INFO] Step 605/874 | Loss: 0.6895 | CE: 0.6895 | LR: 1.14e-05 | Epoch: 2/2 |
| 2026-06-22 19:00:20,714 [INFO] Step 610/874 | Loss: 0.6508 | CE: 0.6508 | LR: 1.10e-05 | Epoch: 2/2 |
| 2026-06-22 19:01:25,988 [INFO] Step 615/874 | Loss: 0.6793 | CE: 0.6793 | LR: 1.07e-05 | Epoch: 2/2 |
| 2026-06-22 19:02:29,223 [INFO] Step 620/874 | Loss: 0.6470 | CE: 0.6470 | LR: 1.03e-05 | Epoch: 2/2 |
| 2026-06-22 19:03:34,145 [INFO] Step 625/874 | Loss: 0.7125 | CE: 0.7125 | LR: 9.90e-06 | Epoch: 2/2 |
| 2026-06-22 19:04:40,433 [INFO] Step 630/874 | Loss: 0.6656 | CE: 0.6656 | LR: 9.54e-06 | Epoch: 2/2 |
| 2026-06-22 19:05:47,144 [INFO] Step 635/874 | Loss: 0.6328 | CE: 0.6328 | LR: 9.18e-06 | Epoch: 2/2 |
| 2026-06-22 19:06:54,351 [INFO] Step 640/874 | Loss: 0.6952 | CE: 0.6952 | LR: 8.82e-06 | Epoch: 2/2 |
| 2026-06-22 19:08:01,812 [INFO] Step 645/874 | Loss: 0.6720 | CE: 0.6720 | LR: 8.47e-06 | Epoch: 2/2 |
| 2026-06-22 19:09:05,437 [INFO] Step 650/874 | Loss: 0.6587 | CE: 0.6587 | LR: 8.13e-06 | Epoch: 2/2 |
| 2026-06-22 19:10:10,727 [INFO] Step 655/874 | Loss: 0.6706 | CE: 0.6706 | LR: 7.79e-06 | Epoch: 2/2 |
| 2026-06-22 19:11:17,984 [INFO] Step 660/874 | Loss: 0.7082 | CE: 0.7082 | LR: 7.45e-06 | Epoch: 2/2 |
| 2026-06-22 19:12:21,738 [INFO] Step 665/874 | Loss: 0.6731 | CE: 0.6731 | LR: 7.13e-06 | Epoch: 2/2 |
| 2026-06-22 19:13:28,847 [INFO] Step 670/874 | Loss: 0.6692 | CE: 0.6692 | LR: 6.81e-06 | Epoch: 2/2 |
| 2026-06-22 19:14:32,901 [INFO] Step 675/874 | Loss: 0.6216 | CE: 0.6216 | LR: 6.49e-06 | Epoch: 2/2 |
| 2026-06-22 19:15:40,203 [INFO] Step 680/874 | Loss: 0.6764 | CE: 0.6764 | LR: 6.18e-06 | Epoch: 2/2 |
| 2026-06-22 19:16:43,589 [INFO] Step 685/874 | Loss: 0.6857 | CE: 0.6857 | LR: 5.88e-06 | Epoch: 2/2 |
| 2026-06-22 19:17:47,913 [INFO] Step 690/874 | Loss: 0.6641 | CE: 0.6641 | LR: 5.59e-06 | Epoch: 2/2 |
| 2026-06-22 19:18:54,829 [INFO] Step 695/874 | Loss: 0.6673 | CE: 0.6673 | LR: 5.30e-06 | Epoch: 2/2 |
| 2026-06-22 19:19:59,148 [INFO] Step 700/874 | Loss: 0.6734 | CE: 0.6734 | LR: 5.02e-06 | Epoch: 2/2 |
| 2026-06-22 19:21:03,180 [INFO] Step 705/874 | Loss: 0.6858 | CE: 0.6858 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:22:07,855 [INFO] Step 710/874 | Loss: 0.6633 | CE: 0.6633 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:23:11,988 [INFO] Step 715/874 | Loss: 0.6516 | CE: 0.6516 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:24:18,106 [INFO] Step 720/874 | Loss: 0.6375 | CE: 0.6375 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:25:23,269 [INFO] Step 725/874 | Loss: 0.6936 | CE: 0.6936 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:26:28,035 [INFO] Step 730/874 | Loss: 0.6723 | CE: 0.6723 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:27:32,438 [INFO] Step 735/874 | Loss: 0.6811 | CE: 0.6811 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:28:37,581 [INFO] Step 740/874 | Loss: 0.7179 | CE: 0.7179 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:29:41,822 [INFO] Step 745/874 | Loss: 0.6541 | CE: 0.6541 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:30:47,379 [INFO] Step 750/874 | Loss: 0.6483 | CE: 0.6483 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:31:50,610 [INFO] Step 755/874 | Loss: 0.6943 | CE: 0.6943 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:32:55,945 [INFO] Step 760/874 | Loss: 0.6906 | CE: 0.6906 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:34:01,473 [INFO] Step 765/874 | Loss: 0.6776 | CE: 0.6776 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:35:08,560 [INFO] Step 770/874 | Loss: 0.7123 | CE: 0.7123 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:36:16,040 [INFO] Step 775/874 | Loss: 0.6830 | CE: 0.6830 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:37:21,682 [INFO] Step 780/874 | Loss: 0.6902 | CE: 0.6902 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:38:24,254 [INFO] Step 785/874 | Loss: 0.6839 | CE: 0.6839 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:39:30,320 [INFO] Step 790/874 | Loss: 0.6676 | CE: 0.6676 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:40:37,677 [INFO] Step 795/874 | Loss: 0.6743 | CE: 0.6743 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:41:39,133 [INFO] Step 800/874 | Loss: 0.7150 | CE: 0.7150 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:42:44,981 [INFO] Step 805/874 | Loss: 0.7008 | CE: 0.7008 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:43:51,776 [INFO] Step 810/874 | Loss: 0.6678 | CE: 0.6678 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:44:56,521 [INFO] Step 815/874 | Loss: 0.6620 | CE: 0.6620 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:46:02,087 [INFO] Step 820/874 | Loss: 0.7109 | CE: 0.7109 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:47:09,600 [INFO] Step 825/874 | Loss: 0.6774 | CE: 0.6774 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:48:10,324 [INFO] Step 830/874 | Loss: 0.7153 | CE: 0.7153 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:49:16,199 [INFO] Step 835/874 | Loss: 0.6919 | CE: 0.6919 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:50:21,486 [INFO] Step 840/874 | Loss: 0.6811 | CE: 0.6811 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:51:24,706 [INFO] Step 845/874 | Loss: 0.6961 | CE: 0.6961 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:52:29,023 [INFO] Step 850/874 | Loss: 0.6782 | CE: 0.6782 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:53:35,015 [INFO] Step 855/874 | Loss: 0.7270 | CE: 0.7270 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:54:41,082 [INFO] Step 860/874 | Loss: 0.7041 | CE: 0.7041 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:55:46,639 [INFO] Step 865/874 | Loss: 0.7503 | CE: 0.7503 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 19:56:51,604 [INFO] Step 870/874 | Loss: 0.7782 | CE: 0.7782 | LR: 5.00e-06 | Epoch: 2/2 |
| 2026-06-22 20:00:50,496 [INFO] Step 874 | Eval Loss: 1.5088 |
| 2026-06-22 20:00:51,631 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/epoch_002.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 20:00:52,641 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/epoch_002.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/epochs/epoch_002_step_874.pt |
| 2026-06-22 20:01:18,459 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/config.yaml -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/config.yaml |
| 2026-06-22 20:01:18,541 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 20:01:18,595 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/train.log -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/train.log |
| 2026-06-22 20:01:18,962 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/checkpoint_manifest.json -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/checkpoint_manifest.json |
| 2026-06-22 20:01:19,039 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 20:01:19,459 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/best.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/best.pt |
| 2026-06-22 20:01:24,307 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 20:01:24,369 [INFO] Deleted old local epoch checkpoint: runs/h200_llm2seq_phase2_lora_encoder/epoch_001.pt |
| 2026-06-22 20:01:26,082 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/checkpoint_874.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 20:01:28,717 [INFO] Saved trainable-only checkpoint to runs/h200_llm2seq_phase2_lora_encoder/final.pt (519 model tensors, stores_base_encoder_weights=False) |
| 2026-06-22 20:01:29,544 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/best.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/best.pt |
| 2026-06-22 20:01:34,598 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 20:01:34,990 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/final.pt -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/final.pt |
| 2026-06-22 20:01:46,592 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/config.yaml -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/config.yaml |
| 2026-06-22 20:01:46,667 [WARNING] No files have been modified since last commit. Skipping to prevent empty commit. |
| 2026-06-22 20:01:46,722 [INFO] Uploading to HF: runs/h200_llm2seq_phase2_lora_encoder/train.log -> BienKieu/llm2seq-wikilingua/checkpoints/h200_phase2_lora_encoder/train.log |
|
|
