---
language:
- en
tags:
- text-to-sql
- sql
- llama-3.2
- postgresql
- gguf
- text-generation
license: llama3.2
base_model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit
---

# Natural Language to SQL (NL2SQL) Llama 3.2 3B - GGUF

This repository contains a fine-tuned version of `unsloth/Llama-3.2-3B-Instruct-bnb-4bit`, specifically trained to translate natural language questions into accurate, executable PostgreSQL queries. 

The model was fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) for 2x faster training and is provided in the **GGUF** format, making it highly optimized for local inference on consumer hardware using tools like [Ollama](https://ollama.com/) or [llama.cpp](https://github.com/ggerganov/llama.cpp).

## Model Details
*   **Base Model:** [unsloth/Llama-3.2-3B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit)
*   **Task:** Text-to-SQL (focusing on PostgreSQL syntax)
*   **Architecture:** Llama 3
*   **Format:** GGUF
*   **Quantization:** `Q4_K_M` (4-bit quantization. This provides an excellent balance between memory usage, inference speed, and model quality).

## How to run with Ollama

1. Download the `saved_finetuned_q4_k_m.gguf` file to your local machine.
2. Create a file named `Modelfile` in the same directory with the following content:

```dockerfile
FROM ./saved_finetuned_q4_k_m.gguf

SYSTEM """You are SQL-Llama, a specialized Text-to-SQL assistant. 
For all inputs, you MUST output ONLY valid SQL code. Do not include markdown blocks or conversational filler."""

TEMPLATE """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""
```

3. Build and run the model in your terminal:

```bash
# Build the model
ollama create natural2sql -f Modelfile

# Run the model
ollama run natural2sql
```

## Prompt Format
To get the best results, provide the database schema as `Context` and your inquiry as `Question`:

```text
Context: CREATE TABLE sales (transaction_id INT, product_name TEXT, amount DECIMAL, sale_date DATE, region TEXT);
Question: What was the total revenue from the 'North' region for transactions occurring after January 2023?
```

## License
The base model weights are subject to the [Meta Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE). Any custom datasets or fine-tuning code associated with this project are provided under the MIT License (see `LICENSE` file).