Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ We evaluate the code solution generation ability of TLLM on three benchmarks: Wi
|
|
| 16 |
| TaPEX | 38.5 | β | β | β | 83.9 | 15.0 | / | 45.8 |
|
| 17 |
| TaPas | 31.5 | β | β | β | 74.2 | 23.1 | / | 42.92 |
|
| 18 |
| TableLlama | 24.0 | 22.2 | 18.9 | 6.4 | 43.7 | 9.0 | / | 20.7 |
|
| 19 |
-
| GPT3.5 | 58.5 |<ins>72.1</ins>| 71.2 | 60.8 | 81.7
|
| 20 |
| GPT4 |**74.1**|**77.1**|**78.4**|**69.5** | 84.0 | 69.5 | 77.8 | **75.8**|
|
| 21 |
| Llama2-Chat (13B) | 48.8 | 49.6 | 67.7 | 61.5 | β | β | β | 56.9 |
|
| 22 |
| CodeLlama (13B) | 43.4 | 47.2 | 57.2 | 49.7 | 38.3 | 21.9 | 47.6 | 43.6 |
|
|
@@ -33,17 +33,17 @@ The prompts we used for generating code solutions and text answers are introduce
|
|
| 33 |
### Code Solution
|
| 34 |
The prompt template for the insert, delete, update, query, and plot operations on a single table.
|
| 35 |
```
|
| 36 |
-
Below are the first few lines of a CSV file. You need to write a Python program to solve the provided question.
|
| 37 |
|
| 38 |
Header and first few lines of CSV file:
|
| 39 |
{csv_data}
|
| 40 |
|
| 41 |
-
Question: {question}
|
| 42 |
```
|
| 43 |
|
| 44 |
The prompt template for the merge operation on two tables.
|
| 45 |
```
|
| 46 |
-
Below are the first few lines two CSV file. You need to write a Python program to solve the provided question.
|
| 47 |
|
| 48 |
Header and first few lines of CSV file 1:
|
| 49 |
{csv_data1}
|
|
@@ -51,7 +51,7 @@ Header and first few lines of CSV file 1:
|
|
| 51 |
Header and first few lines of CSV file 2:
|
| 52 |
{csv_data2}
|
| 53 |
|
| 54 |
-
Question: {question}
|
| 55 |
```
|
| 56 |
|
| 57 |
The csv_data field is filled with the first few lines of your provided table file. Below is an example:
|
|
@@ -67,7 +67,7 @@ I,0.33,0.255,0.08,0.205,0.0895,0.0395,0.055,7
|
|
| 67 |
### Text Answer
|
| 68 |
The prompt template for direct text answer generation on short tables.
|
| 69 |
````
|
| 70 |
-
Offer a thorough and accurate solution that directly addresses the Question outlined in the [Question].
|
| 71 |
### [Table Text]
|
| 72 |
{table_descriptions}
|
| 73 |
|
|
@@ -79,5 +79,5 @@ Offer a thorough and accurate solution that directly addresses the Question outl
|
|
| 79 |
### [Question]
|
| 80 |
{question}
|
| 81 |
|
| 82 |
-
### [Solution]
|
| 83 |
````
|
|
|
|
| 16 |
| TaPEX | 38.5 | β | β | β | 83.9 | 15.0 | / | 45.8 |
|
| 17 |
| TaPas | 31.5 | β | β | β | 74.2 | 23.1 | / | 42.92 |
|
| 18 |
| TableLlama | 24.0 | 22.2 | 18.9 | 6.4 | 43.7 | 9.0 | / | 20.7 |
|
| 19 |
+
| GPT3.5 | 58.5 |<ins>72.1</ins>| 71.2 | 60.8 | 81.7 | 67.4 | 77.1 | 69.8 |
|
| 20 |
| GPT4 |**74.1**|**77.1**|**78.4**|**69.5** | 84.0 | 69.5 | 77.8 | **75.8**|
|
| 21 |
| Llama2-Chat (13B) | 48.8 | 49.6 | 67.7 | 61.5 | β | β | β | 56.9 |
|
| 22 |
| CodeLlama (13B) | 43.4 | 47.2 | 57.2 | 49.7 | 38.3 | 21.9 | 47.6 | 43.6 |
|
|
|
|
| 33 |
### Code Solution
|
| 34 |
The prompt template for the insert, delete, update, query, and plot operations on a single table.
|
| 35 |
```
|
| 36 |
+
[INST]Below are the first few lines of a CSV file. You need to write a Python program to solve the provided question.
|
| 37 |
|
| 38 |
Header and first few lines of CSV file:
|
| 39 |
{csv_data}
|
| 40 |
|
| 41 |
+
Question: {question}[/INST]
|
| 42 |
```
|
| 43 |
|
| 44 |
The prompt template for the merge operation on two tables.
|
| 45 |
```
|
| 46 |
+
[INST]Below are the first few lines two CSV file. You need to write a Python program to solve the provided question.
|
| 47 |
|
| 48 |
Header and first few lines of CSV file 1:
|
| 49 |
{csv_data1}
|
|
|
|
| 51 |
Header and first few lines of CSV file 2:
|
| 52 |
{csv_data2}
|
| 53 |
|
| 54 |
+
Question: {question}[/INST]
|
| 55 |
```
|
| 56 |
|
| 57 |
The csv_data field is filled with the first few lines of your provided table file. Below is an example:
|
|
|
|
| 67 |
### Text Answer
|
| 68 |
The prompt template for direct text answer generation on short tables.
|
| 69 |
````
|
| 70 |
+
[INST]Offer a thorough and accurate solution that directly addresses the Question outlined in the [Question].
|
| 71 |
### [Table Text]
|
| 72 |
{table_descriptions}
|
| 73 |
|
|
|
|
| 79 |
### [Question]
|
| 80 |
{question}
|
| 81 |
|
| 82 |
+
### [Solution][INST/]
|
| 83 |
````
|