Update README.md

README.md

@@ -12,11 +12,11 @@ tags:
 - llama
 ---
 
-# Model Description
+# Introduction
 
 CodeRM-8B is a small yet powerful model designed to enable efficient and high-quality unit test generation. 
-It is trained based on **Llama3.1-8B-Instruct** on a dataset of 60k high-quality synthetic Python unit tests. 
-These unit tests are derived from two well-regarded code instruction tuning datasets: 
+It is trained on a dataset of 60k high-quality synthetic Python unit tests using Llama3.1-70B-Instruct. 
+These unit tests are synthesized based on two well-regarded code instruction tuning datasets: 
 [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) and the 
 training set of [TACO](https://huggingface.co/datasets/BAAI/TACO).
 The training dataset used for unit test generation is openly available under 
@@ -25,6 +25,65 @@ The training dataset used for unit test generation is openly available under
 For further information and details of training, refer to our paper: 
 "Dynamic Scaling of Unit Tests for Code Reward Modeling" available on arXiv.
 
+# Model Information
+
+The model is trained based on [Llama3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct).
+
+# Performance
+
+## Best-of-N
+
+First, we evaluate the performance of CodeRM-8B using a best-of-N setting. In this setup, an LLM (policy model) generates
+100 candidate code solutions for a given programming problem, while another LLM (reward model) generates 100 unit 
+tests. The optimal code solution is then selected based on majority voting derived from the execution results 
+of these unit tests.
+
+Under this framework, our trained unit test generator demonstrates performance comparable to Llama3.1-70B-Instruct,
+despite having an 8x smaller parameter size. The detailed evaluation results across three well-known benchmarks 
+are as follows:
+
+| Model | Policy: Llama3-8B | Policy: Llama3-70B | Policy: GPT-3.5 | Policy: GPT-4o-mini |
+| :------ | :------ | :------ | :------ | :------ |
+| **Benchmark: HumanEval Plus** |||||
+| Vanilla | 53.58 | 73.74 | 67.83 | 82.96 |
+| Reward: Llama3.1-8B | 66.84 (+13.26) | 77.14 (+3.40) | 76.32 (+8.49) | 83.11 (+0.15) |
+| Reward: Llama3.1-70B | **72.04 (+18.46)** | <u>78.54 (+4.80</u>) | **79.76 (+11.93)** | <u>85.45 (+2.49</u>) |
+| Reward: CodeRM-8B | <u>72.01 (+18.43</u>) | **78.69 (+4.95)** | <u>78.01 (+10.18</u>) | **86.38 (+3.42)** |
+| **Benchmark: MBPP Plus** |||||
+| Vanilla | 49.20 | 69.33 | 70.53 | 71.59 |
+| Reward: Llama3.1-8B | 64.31 (+15.11) | 71.64 (+2.31) | 74.18 (+3.65) | 74.48 (+2.89) |
+| Reward: Llama3.1-70B | <u>65.26 (+16.06</u>) | <u>71.85 (+2.52</u>) | <u>75.72 (+5.19</u>) | <u>74.96 (+3.37</u>) |
+| Reward: CodeRM-8B | **66.71 (+17.51)** | **72.44 (+3.11)** | **75.96 (+5.43)** | **75.20 (+3.61)** |
+| **Benchmark: LiveCodeBench** |||||
+| Vanilla | 11.98 | 25.30 | 20.55 | 34.83 |
+| Reward: Llama3.1-70B | <u>13.28 (+1.30</u>) | **28.46 (+3.16)** | **22.80 (+2.25)** | <u>38.60 (+3.77</u>) |
+| Reward: CodeRM-8B | **15.21 (+3.23)** | <u>27.73 (+2.43</u>)| <u>21.76 (+1.21</u>) | **39.20 (+4.37)** |
+
+## Quality of Unit Test
+
+We evaluate the quality of the unit test generated by CodeRM-8B. As each unit test functions as a classifier to 
+determine correct or incorrect solutions, we first utilize accuracy and F1 score as metrics to assess the 
+classification performance of the unit test. 
+
+We further propose two new metrics to detailed evaluate the possibility of the unit test making incorrect judgments.
+False Acceptance Rate (FAR) measures the probability of wrong solutions being accepted by unit tests.
+False Rejection Rate (FRR) measures the probability of correct solutions being rejected by unit tests.
+The calculation formulas for these four metrics are introduced in Appendix D of the paper.
+
+Below is the quality of individual unit tests and the combination of multiple unit tests on HumanEval Plus, 
+utilizing Llama3.1-8B as the policy model. The top two performances are marked in **bold** and _underlined_.
+
+| **Model**           | **Acc (↑)**   | **F1 (↑)**    | **FAR (↓)**   | **FRR (↓)**   |
+|----------------------|---------------|---------------|---------------|---------------|
+| **Quality of Individual Unit Tests** |               |               |               |               |
+| Llama3.1-8B         | 60.02         | 44.97         | 13.66         | 46.13         |
+| Llama3.1-70B        | **73.65**     | **70.15**     | **11.10**     | **34.51**     |
+| *Model (Ours)*      | <u>69.64</u>       | <u>63.63</u>       | <u>11.17</u>       | <u>38.55</u>       |
+| **Quality of Multiple Unit Tests** |               |               |               |               |
+| Llama3.1-8B         | 74.21         | 74.35         | 20.44         | 30.55         |
+| Llama3.1-70B        | <u>78.30</u>       | <u>78.76</u>       | <u>17.19</u>       | <u>25.97</u>       |
+| *Model (Ours)*      | **80.46**     | **81.27**     | **16.48**     | **22.71**     |
+
 # Prompt Format
 
 ```
@@ -40,6 +99,11 @@ Please add detailed comments to the test cases you write. You do not need to tes
 ```
 
 # Citation
+
 If you find our model helpful, please cite the original paper:
 ```
-```
+```
+
+# Contact
+
+If you have any problems, feel free to raise an issue or reach out to us via email at: <zeyaoma@gmail.com>.