Add Transformers library and pipeline tag

README.md

@@ -1,18 +1,20 @@
 ---
-license: llama2
 datasets:
 - RUCKBReasoning/TableLLM-SFT
 language:
 - en
+license: llama2
 tags:
 - Table
 - QA
 - Code
+pipeline_tag: table-question-answering
+library_name: transformers
 ---
 
 # TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios
 
-| **[Paper](https://arxiv.org/abs/2403.19318)** | **[Training set](https://huggingface.co/datasets/RUCKBReasoning/TableLLM-SFT)** | **[Github](https://github.com/RUCKBReasoning/TableLLM)** | **[Homepage](https://tablellm.github.io/)** |
+| **[Paper](https://arxiv.org/abs/2403.19318)** | **[Training set](https://huggingface.co/datasets/RUCKBReasoning/TableLLM-SFT)** | **[Github](https://github.com/TableLLM/TableLLM)** | **[Homepage](https://tablellm.github.io/)** |
 
 We present **TableLLM**, a powerful large language model designed to handle tabular data manipulation tasks efficiently, whether they are embedded in spreadsheets or documents, meeting the demands of real office scenarios. The TableLLM series encompasses two distinct scales: [TableLLM-7B](https://huggingface.co/RUCKBReasoning/TableLLM-7b) and [TableLLM-13B](https://huggingface.co/RUCKBReasoning/TableLLM-13b), which are fine-tuned based on [CodeLlama-7b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf) and [CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf).
 
@@ -92,4 +94,67 @@ The prompt template for direct text answer generation on short tables.
 ### [Solution][INST/]
 ````
 
-For more details about how to use TableLLM, please refer to our GitHub page: <https://github.com/TableLLM/TableLLM>
+## Environment Setup
+
+Install the requirements with pip:
+```
+pip install -r requirements.txt
+```
+
+## Inference
+The inference results of TableLLM are provided in ```inference/results``` folder. You can also obtain the inference result by yourself. The example commands of spreadsheet-embedded tabular data (e.g., WikiSQL) and document-embedded tabular data (e.g., WTQ) are shown below:
+```
+cd inference
+
+python inference_code.py --dataset wikisql --model_path TableLLM-13b
+
+python inference_text.py --dataset wtq --model_path TableLLM-13b
+```
+
+## Evaluation
+The python code in ```evaluation``` folder is used for reproducing evaluation results. For code generation benchmarks, you can run the following command to reproduce the result of TableLLM-13b on WikiSQL:
+```
+cd evaluation/wikisql-eval
+tar -zxvf csv_tables.tar.gz 
+python eval.py --infer_data ../../inference/results/TableLLM-13b/Infer_wikisql.jsonl
+```
+
+For text generation, we use [CritiqueLLM](https://github.com/thu-coai/CritiqueLLM) for judgement. We also provide the judgement results running by ourselves. You can obtain it in ```inference/results``` folder and reproduce the results using the following command:
+```
+cd evaluation/text-eval
+python get_sum_grade.py --grade_data ../../inference/results/TableLLM-13b/Grade_wtq.jsonl
+```
+
+## Deployment
+You can use the code in ```deployment``` folder as the frontend and backend for deploying TableLLM.
+
+![platform](images/platform.png)
+
+Deploy TableLLM using vllm. Remember to modify the PORT and MODEL_PATH in the script and ```config.json```.
+```
+cd deployment
+bash scripts/deploy_tablellm.sh
+```
+
+Install mongodb and change the username and password to yours in ```config.json```. Prepare the default tables and questions:
+```
+bash prepare_default.sh
+```
+
+Deploy the streamlit app:
+```
+streamlit run streamlit.py --server.port PORT
+```
+
+## Citation
+```
+@article{zhang2024tablellm,
+  title={TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios},
+  author={Zhang, Xiaokang and Zhang, Jing and Ma, Zeyao and Li, Yang and Zhang, Bohan and Li, Guanlin and Yao, Zijun and Xu, Kangli and Zhou, Jinchang and Zhang-Li, Daniel and others},
+  journal={arXiv preprint arXiv:2403.19318},
+  year={2024}
+}
+```
+
+## Contact
+If you have any questions, we encourage you to either create Github issues or get in touch with us at <zhang2718@ruc.edu.cn>, <zeyaoma@ruc.edu.cn>, or <zhang-jing@ruc.edu.cn>.