Create README.md

README.md

@@ -0,0 +1,39 @@
+---
+license: apache-2.0
+datasets:
+- EleutherAI/muInstruct
+- camel-ai/math
+language:
+- en
+tags:
+- math
+---
+
+`llemma_7b_muinstruct_camelmath` is an instruction-following finetune of [Llemma 7B](https://huggingface.co/EleutherAI/llemma_7b), trained on the [μInstruct](https://huggingface.co/datasets/EleutherAI/muInstruct) and [camel-ai/math](https://huggingface.co/datasets/camel-ai/math) datasets.
+
+## Input Formatting
+Format input queries as follows:
+```
+input_text = f"Input:{input}\n\nResponse:"
+```
+
+Note that due to an error during training, this model's end-of-sequence token ID is `0` instead of the `2` which is standard for Llama-2 based models. Inference APIs should handle this automatically by reading this repo's `config.json`, but be aware of this difference if you are doing token surgery.
+
+## Evals
+`
+llemma_7b_muinstruct_camelmath` compares favorably to other 7B parameter models on the [Hungarian Math Exam](https://huggingface.co/datasets/keirp/hungarian_national_hs_finals_exam/blob/main/README.md). It surpasses the few-shot performance of Llemma 7B whilst being the strongest Llama-2 7B based model. 
+
+| Model                                                                          | Exam Score |
+| ------------------------------------------------------------------------------ | ---------- |
+| [Code Llama 7B](https://huggingface.co/codellama/CodeLlama-7b-hf) (few-shot)   | 8\%        |
+| [MetaMath 7B](https://huggingface.co/meta-math/MetaMath-7B-V1.0)               | 20\%       |
+| [MAmmoTH 7B](https://huggingface.co/TIGER-Lab/MAmmoTH-7B)                      | 17\%       | 
+| [MAmmoTH Coder 7B](https://huggingface.co/TIGER-Lab/MAmmoTH-Coder-7B)          | 11\%       | 
+| [Llemma 7B](https://huggingface.co/EleutherAI/llemma_7b) (few-shot)            | 23\%       | 
+| [Llemma_7B_muinstruct_camelmath]                                               | 25\%       | 
+| -                                                                              | -          | 
+| [Mistral 7B](https://huggingface.co/mistralai/Mistral-7B-v0.1) (few-shot)      | 22\%       | 
+| [MetaMath Mistral 7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)    | 29\%       | 
+| [OpenChat 3.5](https://huggingface.co/openchat/openchat_3.5)                   | 37\%       | 
+
+