Improve model card: Add HF paper link and fix code snippet

README.md

@@ -1,15 +1,15 @@
 ---
 base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
+language:
+- en
 library_name: transformers
+license: apache-2.0
 pipeline_tag: text-generation
 tags:
 - base_model:adapter:deepseek-ai/DeepSeek-R1-Distill-Llama-8B
 - lora
 - transformers
 - reward-model
-license: apache-2.0
-language:
-- en
 ---
 
 # gPRM-8B
@@ -18,7 +18,8 @@ language:
 This model is a generative outcome reward model finetuned from [DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B), and the [training data](https://huggingface.co/datasets/dongboklee/train_gPRM) is generated by [QwQ-32B](https://huggingface.co/Qwen/QwQ-32B) on [this data](https://huggingface.co/datasets/dongboklee/train).
 
 For details:
-- **Paper:** [Rethinking Reward Models for Multi-Domain Test-Time Scaling](https://arxiv.org/abs/2510.00492)
+- **Paper:** [Rethinking Reward Models for Multi-Domain Test-Time Scaling](https://huggingface.co/papers/2510.00492) (Hugging Face Paper)
+- **arXiv Paper:** [Rethinking Reward Models for Multi-Domain Test-Time Scaling](https://arxiv.org/abs/2510.00492)
 - **Repository:** [https://github.com/db-Lee/Multi-RM](https://github.com/db-Lee/Multi-RM)
 
 
@@ -40,22 +41,40 @@ model.eval()
 model.to(device)
 
 # prompt formatting
-question = 'Question: In Python 3, which of the following function convert a string to an int in python?\nA. short(x)\nB. float(x)\nC. integer(x [,base])\nD. double(x)\nE. int(x [,base])\nF. long(x [,base] )\nG. num(x)\nH. str(x)\nI. char(x)\nJ. digit(x [,base])'
+question = 'Question: In Python 3, which of the following function convert a string to an int in python?
+A. short(x)
+B. float(x)
+C. integer(x [,base])
+D. double(x)
+E. int(x [,base])
+F. long(x [,base] )
+G. num(x)
+H. str(x)
+I. char(x)
+J. digit(x [,base])'
 solution = ["To convert a string to an integer in Python 3, we use the built-in function int().",
             "The int() function takes two arguments: the string to be converted and an optional base (default is 10, which is for decimal).",
-            "For example: int(\"123\", 10) converts the string \"123\" to the integer 123.",
+            "For example: int(\\\"123\\\", 10) converts the string \\\"123\\\" to the integer 123.",
             "Looking at the options, we can see that the correct function is option E: int(x [,base]).",
             "The answer is (E)."]
 category_name = "computer science"
-prefix = "\n\n".join(solution)
-steps = [ f"Step {str(i+1)}: {step}" for i, step in enumerate(steps) ]
-prefix = "\n".join(steps)
+steps = [ f"Step {str(i+1)}: {step}" for i, step in enumerate(solution) ]
+prefix = "
+".join(steps)
 
 # Create the prompt
 prompt_text = (
-    f"You are given a {category_name} problem and a proposed step-by-step solution:\n\n"
-    f"[{category_name.capitalize()} Problem]\n{question}\n\n"
-    f"[Solution]\n{prefix}\n\n"
+    f"You are given a {category_name} problem and a proposed step-by-step solution:\
+\
+"
+    f"[{category_name.capitalize()} Problem]\
+{question}\
+\
+"
+    f"[Solution]\
+{prefix}\
+\
+"
     "Review and critique each step in the proposed solution to determine whether each step is correct. If the solution is incomplete, only verify the provided steps."
 )
 
@@ -78,7 +97,10 @@ with torch.no_grad():
     )
 
 # compute reward
-logits = outputs.logits[0, -2, :]
+# `outputs.scores` contains the logits for each generated token.
+# The model is expected to end its generation with " Yes" or " No".
+# We take the logits of the very last generated token to compute the reward.
+logits = outputs.scores[-1][0]
 yes_logit, no_logit = logits[yes_id].item(), logits[no_id].item()
 reward = math.exp(yes_logit) / (math.exp(yes_logit) + math.exp(no_logit))
 ```