| [2025-03-23 15:04:12,688][__main__][INFO] - cache_dir: /media/data/tmp | |
| dataset: | |
| name: kamel-usp/aes_enem_dataset | |
| split: JBCS2025 | |
| training_params: | |
| seed: 42 | |
| num_train_epochs: 20 | |
| logging_steps: 100 | |
| metric_for_best_model: QWK | |
| bf16: true | |
| post_training_results: | |
| model_path: /workspace/jbcs2025/outputs/2025-03-17/22-47-57 | |
| experiments: | |
| model: | |
| name: microsoft/phi-4 | |
| type: phi4_classification_lora | |
| num_labels: 6 | |
| output_dir: ./results/phi4-balanced/C1 | |
| logging_dir: ./logs/phi4-balanced/C1 | |
| best_model_dir: ./results/phi4-balanced/C1/best_model | |
| lora_r: 8 | |
| lora_dropout: 0.05 | |
| lora_alpha: 16 | |
| lora_target_modules: all-linear | |
| dataset: | |
| grade_index: 0 | |
| training_id: phi4-balanced-C1 | |
| training_params: | |
| weight_decay: 0.01 | |
| warmup_ratio: 0.1 | |
| learning_rate: 5.0e-05 | |
| train_batch_size: 1 | |
| eval_batch_size: 16 | |
| gradient_accumulation_steps: 16 | |
| gradient_checkpointing: false | |
| [2025-03-23 15:04:12,690][__main__][INFO] - Starting the Fine Tuning training process. | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file vocab.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/vocab.json | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file merges.txt from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/merges.txt | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file tokenizer.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/tokenizer.json | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file added_tokens.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/added_tokens.json | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file special_tokens_map.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/special_tokens_map.json | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file tokenizer_config.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/tokenizer_config.json | |
| [2025-03-23 15:04:15,506][transformers.tokenization_utils_base][INFO] - loading file chat_template.jinja from cache at None | |
| [2025-03-23 15:04:15,635][__main__][INFO] - Tokenizer function parameters- Padding:longest; Truncation: False | |
| [2025-03-23 15:04:15,839][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:04:15,839][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "id2label": { | |
| "0": 0, | |
| "1": 40, | |
| "2": 80, | |
| "3": 120, | |
| "4": 160, | |
| "5": 200 | |
| }, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "label2id": { | |
| "0": 0, | |
| "40": 1, | |
| "80": 2, | |
| "120": 3, | |
| "160": 4, | |
| "200": 5 | |
| }, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:04:15,860][transformers.modeling_utils][INFO] - loading weights file model.safetensors from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/model.safetensors.index.json | |
| [2025-03-23 15:04:15,861][transformers.modeling_utils][INFO] - Will use torch_dtype=torch.bfloat16 as defined in model's config object | |
| [2025-03-23 15:04:15,861][transformers.modeling_utils][INFO] - Instantiating Phi3ForSequenceClassification model under default dtype torch.bfloat16. | |
| [2025-03-23 15:04:37,315][transformers.modeling_utils][INFO] - Some weights of the model checkpoint at microsoft/phi-4 were not used when initializing Phi3ForSequenceClassification: ['lm_head.weight'] | |
| - This IS expected if you are initializing Phi3ForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). | |
| - This IS NOT expected if you are initializing Phi3ForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). | |
| [2025-03-23 15:04:37,316][transformers.modeling_utils][WARNING] - Some weights of Phi3ForSequenceClassification were not initialized from the model checkpoint at microsoft/phi-4 and are newly initialized: ['score.weight'] | |
| You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. | |
| [2025-03-23 15:04:42,122][__main__][INFO] - None | |
| [2025-03-23 15:04:42,123][transformers.training_args][INFO] - PyTorch: setting up devices | |
| [2025-03-23 15:04:42,177][__main__][INFO] - Total steps: 620. Number of warmup steps: 62 | |
| [2025-03-23 15:04:42,183][transformers.trainer][INFO] - You have loaded a model on multiple GPUs. `is_model_parallel` attribute will be force-set to `True` to avoid any unexpected behavior such as device placement mismatching. | |
| [2025-03-23 15:04:42,201][transformers.trainer][INFO] - Using auto half precision backend | |
| [2025-03-23 15:04:42,202][transformers.trainer][WARNING] - No label_names provided for model class `PeftModelForSequenceClassification`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead. | |
| [2025-03-23 15:04:42,218][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:04:42,230][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:04:42,230][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:04:42,230][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:05:05,414][transformers][INFO] - {'accuracy': 0.4621212121212121, 'RMSE': 44.17595671307132, 'QWK': -0.25277057297807115, 'HDIV': 0.05303030303030298, 'Macro_F1': 0.1304812834224599, 'Micro_F1': 0.4621212121212121, 'Weighted_F1': 0.31137578998541565, 'Macro_F1_(ignoring_nan)': np.float64(0.32620320855614976)} | |
| [2025-03-23 15:05:05,416][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:05:05,636][transformers.trainer][INFO] - The following columns in the training set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:05:05,666][transformers.trainer][INFO] - ***** Running training ***** | |
| [2025-03-23 15:05:05,666][transformers.trainer][INFO] - Num examples = 500 | |
| [2025-03-23 15:05:05,667][transformers.trainer][INFO] - Num Epochs = 20 | |
| [2025-03-23 15:05:05,667][transformers.trainer][INFO] - Instantaneous batch size per device = 1 | |
| [2025-03-23 15:05:05,667][transformers.trainer][INFO] - Total train batch size (w. parallel, distributed & accumulation) = 16 | |
| [2025-03-23 15:05:05,667][transformers.trainer][INFO] - Gradient Accumulation steps = 16 | |
| [2025-03-23 15:05:05,667][transformers.trainer][INFO] - Total optimization steps = 620 | |
| [2025-03-23 15:05:05,669][transformers.trainer][INFO] - Number of trainable parameters = 27,883,520 | |
| [2025-03-23 15:11:20,753][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:11:20,755][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:11:20,755][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:11:20,755][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:11:43,814][transformers][INFO] - {'accuracy': 0.4772727272727273, 'RMSE': 35.67530340063379, 'QWK': 0.0, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.12923076923076923, 'Micro_F1': 0.4772727272727273, 'Weighted_F1': 0.30839160839160845, 'Macro_F1_(ignoring_nan)': np.float64(0.6461538461538462)} | |
| [2025-03-23 15:11:43,814][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:11:43,816][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-32 | |
| [2025-03-23 15:11:44,193][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:11:44,194][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:18:00,654][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:18:00,657][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:18:00,657][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:18:00,657][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:18:23,632][transformers][INFO] - {'accuracy': 0.49242424242424243, 'RMSE': 35.33390508399614, 'QWK': 0.03629146583498721, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.15774990868135882, 'Micro_F1': 0.49242424242424243, 'Weighted_F1': 0.351078666858528, 'Macro_F1_(ignoring_nan)': np.float64(0.3943747717033971)} | |
| [2025-03-23 15:18:23,632][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:18:23,635][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-64 | |
| [2025-03-23 15:18:23,868][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:18:23,869][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:18:24,533][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-32] due to args.save_total_limit | |
| [2025-03-23 15:24:39,278][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:24:39,280][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:24:39,280][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:24:39,280][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:25:02,286][transformers][INFO] - {'accuracy': 0.49242424242424243, 'RMSE': 31.333978072025612, 'QWK': 0.4662539936102237, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.2550461466497952, 'Micro_F1': 0.49242424242424243, 'Weighted_F1': 0.44255470368357247, 'Macro_F1_(ignoring_nan)': np.float64(0.425076911082992)} | |
| [2025-03-23 15:25:02,286][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:25:02,289][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-96 | |
| [2025-03-23 15:25:02,557][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:25:02,558][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:25:03,210][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-64] due to args.save_total_limit | |
| [2025-03-23 15:31:17,930][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:31:17,932][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:31:17,932][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:31:17,933][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:31:40,845][transformers][INFO] - {'accuracy': 0.6590909090909091, 'RMSE': 26.74231693686086, 'QWK': 0.6705583756345177, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.505355276907001, 'Micro_F1': 0.6590909090909091, 'Weighted_F1': 0.655035860169089, 'Macro_F1_(ignoring_nan)': np.float64(0.6316940961337513)} | |
| [2025-03-23 15:31:40,845][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:31:40,847][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-128 | |
| [2025-03-23 15:31:41,099][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:31:41,100][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:31:41,822][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-96] due to args.save_total_limit | |
| [2025-03-23 15:37:56,610][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:37:56,613][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:37:56,613][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:37:56,613][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:38:19,586][transformers][INFO] - {'accuracy': 0.6742424242424242, 'RMSE': 25.81988897471611, 'QWK': 0.6984548928393421, 'HDIV': 0.0, 'Macro_F1': 0.5295884500497843, 'Micro_F1': 0.6742424242424242, 'Weighted_F1': 0.6732932544830984, 'Macro_F1_(ignoring_nan)': np.float64(0.6619855625622303)} | |
| [2025-03-23 15:38:19,586][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:38:19,588][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-160 | |
| [2025-03-23 15:38:19,821][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:38:19,822][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:38:20,534][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-128] due to args.save_total_limit | |
| [2025-03-23 15:44:35,255][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:44:35,256][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:44:35,256][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:44:35,256][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:44:58,245][transformers][INFO] - {'accuracy': 0.5833333333333334, 'RMSE': 27.633971188310298, 'QWK': 0.5477485316510767, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.30369105025593573, 'Micro_F1': 0.5833333333333334, 'Weighted_F1': 0.5354999307115893, 'Macro_F1_(ignoring_nan)': np.float64(0.5061517504265596)} | |
| [2025-03-23 15:44:58,246][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:44:58,248][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-192 | |
| [2025-03-23 15:44:58,581][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:44:58,582][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:51:14,206][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:51:14,208][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:51:14,209][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:51:14,209][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:51:37,193][transformers][INFO] - {'accuracy': 0.6590909090909091, 'RMSE': 24.863262420322442, 'QWK': 0.6629280993390747, 'HDIV': 0.0, 'Macro_F1': 0.45253388586721915, 'Micro_F1': 0.6590909090909091, 'Weighted_F1': 0.6151236530024409, 'Macro_F1_(ignoring_nan)': np.float64(0.565667357334024)} | |
| [2025-03-23 15:51:37,193][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:51:37,196][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-224 | |
| [2025-03-23 15:51:37,529][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 15:51:37,530][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 15:51:38,236][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-192] due to args.save_total_limit | |
| [2025-03-23 15:57:53,116][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 15:57:53,118][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 15:57:53,118][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 15:57:53,118][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 15:58:16,064][transformers][INFO] - {'accuracy': 0.553030303030303, 'RMSE': 28.06917861068948, 'QWK': 0.6053357865685374, 'HDIV': 0.0, 'Macro_F1': 0.38989173236304836, 'Micro_F1': 0.553030303030303, 'Weighted_F1': 0.516636553528003, 'Macro_F1_(ignoring_nan)': np.float64(0.48736466545381046)} | |
| [2025-03-23 15:58:16,065][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 15:58:16,068][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-256 | |
| [2025-03-23 15:58:26,827][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-224] due to args.save_total_limit | |
| [2025-03-23 16:04:41,848][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 16:04:41,850][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 16:04:41,850][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 16:04:41,850][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 16:05:04,713][transformers][INFO] - {'accuracy': 0.6742424242424242, 'RMSE': 26.285149626910837, 'QWK': 0.6787909836065573, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.49290131869862563, 'Micro_F1': 0.6742424242424242, 'Weighted_F1': 0.6694937343413603, 'Macro_F1_(ignoring_nan)': np.float64(0.616126648373282)} | |
| [2025-03-23 16:05:04,714][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 16:05:04,716][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-288 | |
| [2025-03-23 16:05:05,401][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 16:05:05,401][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 16:05:06,147][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-256] due to args.save_total_limit | |
| [2025-03-23 16:11:21,262][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 16:11:21,264][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 16:11:21,264][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 16:11:21,264][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 16:11:44,181][transformers][INFO] - {'accuracy': 0.6742424242424242, 'RMSE': 26.285149626910837, 'QWK': 0.6179935012185216, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.4704739704739705, 'Micro_F1': 0.6742424242424242, 'Weighted_F1': 0.6576546349273622, 'Macro_F1_(ignoring_nan)': np.float64(0.5880924630924631)} | |
| [2025-03-23 16:11:44,181][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 16:11:44,184][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-320 | |
| [2025-03-23 16:11:44,464][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 16:11:44,465][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 16:11:45,140][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-288] due to args.save_total_limit | |
| [2025-03-23 16:11:45,183][transformers.trainer][INFO] - | |
| Training completed. Do not forget to share your model on huggingface.co/models =) | |
| [2025-03-23 16:11:45,183][transformers.trainer][INFO] - Loading best model from /workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-160 (score: 0.6984548928393421). | |
| [2025-03-23 16:11:45,419][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/15-04-12/results/phi4-balanced/C1/checkpoint-320] due to args.save_total_limit | |
| [2025-03-23 16:11:45,470][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 16:11:45,471][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 16:11:45,471][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 16:11:45,471][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 16:12:08,400][transformers][INFO] - {'accuracy': 0.6742424242424242, 'RMSE': 25.81988897471611, 'QWK': 0.6984548928393421, 'HDIV': 0.0, 'Macro_F1': 0.5295884500497843, 'Micro_F1': 0.6742424242424242, 'Weighted_F1': 0.6732932544830984, 'Macro_F1_(ignoring_nan)': np.float64(0.6619855625622303)} | |
| [2025-03-23 16:12:08,402][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 16:12:08,403][__main__][INFO] - Training completed successfully. | |
| [2025-03-23 16:12:08,403][__main__][INFO] - Running on Test | |
| [2025-03-23 16:12:08,403][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades. If id, essay_text, essay_year, id_prompt, prompt, reference, supporting_text, grades are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 16:12:08,405][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 16:12:08,405][transformers.trainer][INFO] - Num examples = 138 | |
| [2025-03-23 16:12:08,405][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 16:12:32,927][transformers][INFO] - {'accuracy': 0.6159420289855072, 'RMSE': 27.240208984279956, 'QWK': 0.6748637903107053, 'HDIV': 0.007246376811594235, 'Macro_F1': 0.46569434642829133, 'Micro_F1': 0.6159420289855072, 'Weighted_F1': 0.6184188545576659, 'Macro_F1_(ignoring_nan)': np.float64(0.5821179330353642)} | |
| [2025-03-23 16:12:32,927][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 16:12:32,930][transformers.trainer][INFO] - Saving model checkpoint to ./results/phi4-balanced/C1/best_model | |
| [2025-03-23 16:12:33,185][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 16:12:33,185][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 16:12:33,735][__main__][INFO] - Fine Tuning Finished. | |