Training in progress, step 50

README.md

@@ -1,5 +1,5 @@
 ---
-datasets: DeepMath-103k
+base_model: Thrillcrazyer/Qwen-7B_THIP
 library_name: transformers
 model_name: Qwen-7B_THIP
 tags:
@@ -9,30 +9,60 @@ tags:
 licence: license
 ---
 
-<h1 align= "center"> Reasoning-Aware GRPO using Process Mining </h1>
+# Model Card for Qwen-7B_THIP
 
-<p align="center">
-  <a href="https://pnubaelab.github.io/"><b>BAELAB</b></a>, Pusan National University, Busan, Korea
-</p>
-<p align="center">
-  Taekyhun Park<sup>*</sup> , Yongjae Lee<sup>*</sup>, Hyerim Bae<sup>&dagger;</sup>
-</p>
+This model is a fine-tuned version of [Thrillcrazyer/Qwen-7B_THIP](https://huggingface.co/Thrillcrazyer/Qwen-7B_THIP).
+It has been trained using [TRL](https://github.com/huggingface/trl).
 
+## Quick start
 
+```python
+from transformers import pipeline
 
-<p align="center">
-  <a href="https://github.com/Thrillcrazyer/THIP"><b>🌟 Github</b></a> |
-  <a href="https://huggingface.co/Thrillcrazyer/Qwen-1.5B_THIP"><b>📥 1.5B Download</b></a> |
-  <a href="https://huggingface.co/Thrillcrazyer/Qwen-1.5B_THIP"><b>📥 7B Download</b></a> |
-  <a href="https://arxiv.org/abs/2510.25065"><b>📄 Arxiv Paper Link</b></a> |
-</p>
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="Thrillcrazyer/Qwen-7B_THIP", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
 
-# Abstract
+## Training procedure
 
-Reinforcement learning (RL)-based post-training has been crucial for enabling multi-step reasoning in large reasoning models (LRMs), yet current reward schemes are typically outcome-centric. We propose **PM4GRPO**, a reasoning-aware Group Relative Policy Optimization (GRPO) that augments standard answer/format rewards with signals over the reasoning procedure. To this end, process mining techniques are utilized to compute a scalar conformance reward that measures how closely a policy model's reasoning aligns with the pretrained teacher model. The empirical results on five benchmarks demonstrate that **PM4GRPO** significantly outperforms existing methodologies for GRPO-based post-training. These results highlight that leveraging process mining for reasoning-aware GRPO effectively enhances the reasoning capabilities of policy models.
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/pthpark1/TAC/runs/whtsmb9d) 
 
-# Illustration of PM4GRPO
 
-<div align="center">
-<img src="https://arxiv.org/html/2510.25065v1/x1.png" width="600"/>
-</div>
+This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
+
+### Framework versions
+
+- TRL: 0.27.0
+- Transformers: 4.57.6
+- Pytorch: 2.8.0
+- Datasets: 4.5.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+Cite GRPO as:
+
+```bibtex
+@article{shao2024deepseekmath,
+    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
+    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
+    year         = 2024,
+    eprint       = {arXiv:2402.03300},
+}
+
+```
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

config.json

@@ -52,7 +52,7 @@
   "rope_theta": 10000,
   "sliding_window": null,
   "tie_word_embeddings": false,
-  "transformers_version": "4.57.1",
+  "transformers_version": "4.57.6",
   "use_cache": true,
   "use_mrope": false,
   "use_sliding_window": false,

generation_config.json

@@ -8,5 +8,5 @@
   "pad_token_id": 151643,
   "temperature": 0.6,
   "top_p": 0.95,
-  "transformers_version": "4.57.1"
+  "transformers_version": "4.57.6"
 }

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ae3f733de4131deef1b2cb63aa102b48653bd3bf01ae4ac09870accaadd0d30d
+oid sha256:11a4f5f4ea26f09007057d60a769c6fbef7bc0df001c92d25e4e2dee75b7d569
 size 4877660776

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:62d644447d298c8be54be5ddc441518660ffdae0b655dd19ec793ada3d747b68
+oid sha256:ea807fb19350c86aae87e5e5b162c25aa5b435b2f717a1e07601ee7686a19f74
 size 4932751008

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf676d93247dbf99ff1d1cbca37981160af1c2ed260fdcdef3a410b3d9637623
+oid sha256:3f0f2b7c5ab74e9f0e045b3b1c85860450de81b72725df2045e6f1fdfa9f57e6
 size 4330865200

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:16dbca31381da9cb6c7bc5b23da6f9d6d9a3b5ecf975cdaa1432d71552c17437
+oid sha256:bc85e035063a45c3a98fa44d1b9d4f12ded755bb03c37b702ef13e9fc0cb0e83
 size 1089994880

tokenizer_config.json

@@ -185,11 +185,17 @@
   "eos_token": "<｜end▁of▁sentence｜>",
   "extra_special_tokens": {},
   "legacy": true,
+  "max_length": 512,
   "model_max_length": 16384,
+  "pad_to_multiple_of": null,
   "pad_token": "<｜end▁of▁sentence｜>",
+  "pad_token_type_id": 0,
+  "padding_side": "left",
   "sp_model_kwargs": {},
+  "stride": 0,
   "tokenizer_class": "LlamaTokenizerFast",
   "truncation_side": "left",
+  "truncation_strategy": "longest_first",
   "unk_token": null,
   "use_default_system_prompt": false
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e604235bce63f4814885835e394c83ec87b2108cd8832606c78a234cf03a59e8
-size 8593
+oid sha256:b5b7839c52316e8f9028964b0dce717dc11ebd34c43ebffd1f90fc40998f0bfd
+size 9169