Add model description to README

README.md

@@ -11,4 +11,30 @@ tags:
 - legal
 - portuguese
 - Brazil
----
+---
+
+# Juru: Legal Brazilian Large Language Model from Reputable Sources
+
+This repository hosts the public checkpoints for **Juru-7B**, a Mistral-7B specialised in the Brazilian legal domain. The model was continued pretrained on **1.9 billion** unique tokens from reputable academic and legal sources in Portuguese. For full details on data curation, training, and evaluation, see our paper: <https://arxiv.org/abs/2403.18140>.
+
+## Checkpoints
+
+* Checkpoints were saved every **200** optimization steps up to step **3,800**.  
+* Each 200 step interval adds **~0.4 billion** tokens of continued pretraining.
+* We refer to **Juru-7B** as checkpoint **3,400** (~7.1 billion tokens), which achieved the best score on our Brazilian legal knowledge benchmarks.
+
+> **Note:** The model has **not** been instruction finetuned. For best results, use few-shot inference or perform additional finetuning on your specific task.
+
+## Citation information
+
+```bibtex
+@misc{junior2024jurulegalbrazilianlarge,
+      title={Juru: Legal Brazilian Large Language Model from Reputable Sources}, 
+      author={Roseval Malaquias Junior and Ramon Pires and Roseli Romero and Rodrigo Nogueira},
+      year={2024},
+      eprint={2403.18140},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2403.18140}, 
+}
+```