Update README.md
Browse files
README.md
CHANGED
|
@@ -19,10 +19,10 @@ license: apache-2.0
|
|
| 19 |
|
| 20 |
|
| 21 |
### 2. Evaluation Results
|
|
|
|
|
|
|
| 22 |
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
### 3. License
|
| 26 |
This code repository is licensed under the MIT License.
|
| 27 |
The use of UT-LM models is subject to the Model License. UT-LM supports commercial use.
|
| 28 |
|
|
|
|
| 19 |
|
| 20 |
|
| 21 |
### 2. Evaluation Results
|
| 22 |
+
Based on the $MLFT$ and the $SFRL$ training framework, we trained two unit test models (UT-LM-7b and UT-LM-33b), one with 7B parameters \footnote{https://huggingface.co/Arain/UT-LM-7B} and the other with 33B parameters \footnote{https://huggingface.co/Arain/UT-LM-33B}.
|
| 23 |
+
| Model | Java (Total: 98 methods from 26 repositories) | | | | | | Python (Total: 449 methods from 20 repositories) | | | | |---------------------------|-----------------------------------------------|-----|-----|-------|-----|-----|-------------------------------------------------|-----|-------|-----| | | **Test Completion** | | | **Test Generation** | | | **Test Completion** | | **Test Generation** | | | | Compile Error | Test Error | Pass | Compile Error | Test Error | Pass | Test Error | Pass | Test Error | Pass | | **GPT-3.5** | 77 | 2 | 19 | 77 | 8 | 13 | 396 | 53 | 418 | 31 | | **CodeLlama-13b-Instruct**| 76 | 7 | 15 | 66 | 18 | 14 | 391 | 58 | 421 | 27 | | **Deepseek-coder-7b-instruct** | 80 | 10 | 8 | 79 | 7 | 12 | 363 | 81 | 384 | 64 | | **Deepseek-coder-33b-instruct**| 76 | 9 | 13 | 72 | 14 | 12 | 338 | 107 | 367 | 81 | | **GPT-4** | 42 | 15 | 41 | 52 | 11 | 35 | 342 | 104 | 328 | 118 | | **WizardCoder-15b** | 53 | 19 | 26 | 63 | 18 | 17 | 369 | 77 | 402 | 45 | | **UT-LM-7b** | 45 | 14 | 39 | 51 | 17 | 30 | 340 | 105 | 333 | 113 | | **UT-LM-33b** | 46 | 13 | 39 | 48 | 16 | 34 | 328 | 117 | 332 | 115 |
|
| 24 |
|
| 25 |
+
### 4. License
|
|
|
|
|
|
|
| 26 |
This code repository is licensed under the MIT License.
|
| 27 |
The use of UT-LM models is subject to the Model License. UT-LM supports commercial use.
|
| 28 |
|