Update README.md
Browse files
README.md
CHANGED
|
@@ -8,3 +8,47 @@ metrics:
|
|
| 8 |
- accuracy
|
| 9 |
pipeline_tag: text-generation
|
| 10 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
- accuracy
|
| 9 |
pipeline_tag: text-generation
|
| 10 |
---
|
| 11 |
+
# How Do Humans Write Code? Large Models Do It the Same Way Too
|
| 12 |
+
|
| 13 |
+
Paper: [https://arxiv.org/pdf/2402.15729](https://arxiv.org/pdf/2402.15729)
|
| 14 |
+
|
| 15 |
+
Code: [https://github.com/seamoke/Human-Think-Language](https://github.com/seamoke/Human-Think-Language)
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
## Introduction
|
| 19 |
+
We introduce HTL, a model which utilizes the complete reasoning process of CoT to enhance PoT. This model was secondarily fine-tuned based on [MAmmoTH-Coder-7B](https://huggingface.co/TIGER-Lab/MAmmoTH-Coder-7B)
|
| 20 |
+
|
| 21 |
+
## Evaluation
|
| 22 |
+
The models are evaluated using open-ended and multiple-choice math problems from several datasets. Here are the results:
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
| **Model** | **GSM** |**GSM-Hard** | **NumGLUE** | **MATH** | **Sim** | **SVAMP** | **MAWPS** | **ASDiV** |
|
| 26 |
+
|---------------------------| ----------|---------------|---------------|-----------|----------|---------- |------------|---------------|
|
| 27 |
+
| **MAmmoTH-Coder-7B** | 59.4 |56.3 | 66.4 |33.4| 45.9 | 70.7 | 91.9 | 69.3 |
|
| 28 |
+
| **TORA** | **72.6** |56.0 | 46.2 |**44.6**| 48.5 | 70.4 | 91.3 | **78.7** |
|
| 29 |
+
| **MAmmoTH-Coder-7B** | 65.7 |**58.3** | **75.1** |34.9| **50.8** | **74.4** | **94.2** | 73.1 |
|
| 30 |
+
|
| 31 |
+
## Prompt Format
|
| 32 |
+
If you want to do HTL:
|
| 33 |
+
```
|
| 34 |
+
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
| 35 |
+
I'd like you to solve this problem in 3 steps:
|
| 36 |
+
1.Answer the question in plain language without writing any code.\n
|
| 37 |
+
2.Output one line of *\n.
|
| 38 |
+
3.Write program code based on the solution process in step 1 to solve the problem.\n
|
| 39 |
+
### Instruction:
|
| 40 |
+
{query}
|
| 41 |
+
Let's write a program.
|
| 42 |
+
### Response:"
|
| 43 |
+
```
|
| 44 |
+
## Citation
|
| 45 |
+
If you use the models, data, or code from this project, please cite the original paper:
|
| 46 |
+
|
| 47 |
+
```
|
| 48 |
+
@article{li2024humans,
|
| 49 |
+
title={How Do Humans Write Code? Large Models Do It the Same Way Too},
|
| 50 |
+
author={Li, Long},
|
| 51 |
+
journal={arXiv preprint arXiv:2402.15729},
|
| 52 |
+
year={2024}
|
| 53 |
+
}
|
| 54 |
+
```
|