Create README.md

README.md

@@ -0,0 +1,54 @@
+---
+license: apache-2.0
+language:
+- en
+library_name: transformers
+---
+
+## Introduction 🎉
+
+A model finetuned specifically for the text-to-SQL tasks. The model is finetuned on mistral 7B with a curated dataset of 100k SQL query generation instructions.
+
+
+## Generate responses
+
+Now that your model is fine-tuned, you're ready to generate responses. You can do this using our generate.py script, which runs inference from the Hugging Face model hub and inference on a specified input. Here's an example of usage:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("budecosystem/sql-millennials-7b")
+model = AutoModelForCausalLM.from_pretrained("budecosystem/sql-millennials-7b")
+
+prompt = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+USER: Create SQL query for the given table schema and question ASSISTANT:"
+
+inputs = tokenizer(prompt, return_tensors="pt")
+sample = model.generate(**inputs, max_length=128)
+print(tokenizer.decode(sample[0]))
+
+```
+
+
+## Training details
+
+The model is trained of 4 A100 80GB for approximately 30hrs. 
+
+| Hyperparameters              | Value  |
+| :----------------------------| :-----: |
+| per_device_train_batch_size  | 4      |
+| gradient_accumulation_steps  | 1      |
+| epoch | 3 |
+| steps | 19206 |
+| learning_rate                | 2e-5   |
+| lr schedular type | cosine |
+| warmup steps | 2000 |
+| optimizer                    | adamw  |
+| fp16                         | True   |
+| GPU                          | 4 A100 80GB |
+
+
+### Acknowledgments
+
+We'd like to thank the open-source community and the researchers whose foundational work laid the path to this  model.