README.md

---
base_model: google/gemma-2b
library_name: peft
license: bsl-1.0
tags:
- code
datasets:
- b-mc2/sql-create-context
language:
- en
pipeline_tag: text2text-generation
---

# Model Card for Model ID

### Model Description

This model is quantized in 8-bit and trained with question and answer pairs for text-to-SQL tasks using the LoRA PEFT method. It serves as a foundation model for further development in Text-to-SQL Retrieval-Augmented Generation (RAG) applications.

- **Developed by:** Lei-bw
- **Model type:** Causal Language Model
- **Language(s) (NLP):** English
- **License:** bsl-1.0
- **Finetuned from model:** google/gemma-2b
- **Device to be used:** NVIDIA GeForce RTX 3080 Ti (12288MiB)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/65600cc95b627f441ad58374/BXun8ig2s9QoASHz6sSP9.png)

## How to Get Started with the Model

1. Apply the access to model google/gemma-2b
You may submit the approval from [google/gemma-2b](https://huggingface.co/google/gemma-2b)
   
2. Login-in
If you are using cli, please use below command to login your huggingface account with your access token key:
```bash
huggingface-cli login
```

OR

If you are using notebook, please use below command to login your huggingface account with your access token key:
```python
from huggingface_hub import login
login()
```

3. Install the required library

```python
peft==0.12.0
transformers==4.44.2
```


4. Get access to the `Lei-bw/text-to-sql-fm` model
```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

config = PeftConfig.from_pretrained("Lei-bw/text-to-sql-fm")
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2b")
model = PeftModel.from_pretrained(base_model, "Lei-bw/text-to-sql-fm")
tokenizer = AutoTokenizer.from_pretrained("Lei-bw/text-to-sql-fm")
```

![image/png](https://cdn-uploads.huggingface.co/production/uploads/65600cc95b627f441ad58374/jPZ46TQAD_zD9G-Y9LypK.png)


5. Example to use this model:
```python
import torch

# Example usage
eval_prompt = """
How many seniors in the departments are younger than 28?
"""

model_input = tokenizer(eval_prompt, return_tensors="pt")

model.eval()
with torch.no_grad():
    print(tokenizer.decode(model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))
```

## Training Details

### Training Data

The model was trained on the b-mc2/sql-create-context dataset, which contains question and answer pairs for SQL generation tasks.


#### Training Hyperparameters

    •	Training regime: bf16 mixed precision
	•	Batch size: 16 
	•	Gradient accumulation steps: 4
	•	Warmup steps: 50
	•	Number of epochs: 2
	•	Learning rate: 2e-4
	•	Weight decay: 0.01
	•	Optimizer: AdamW
	•	Learning rate scheduler: Linear

#### Hardware

- **Hardware Type:** NVIDIA A100
- **GPU RAM:** 40 GB


### Framework versions

- PEFT 0.12.0