README.md

---
base_model: mistralai/Mistral-7B-Instruct-v0.3
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
- en
tags:
- lora
- peft
- text-to-sql
- dynquant
datasets:
- gretelai/synthetic_text_to_sql
- Salesforce/wikisql
- b-mc2/sql-create-context
---

# Mistral-7B-Instruct-v0.3 text-to-SQL LoRA

The rank-32 LoRA adapter that produced
[`VikramPal/mistral-7b-instruct-v0.3-text2sql-bf16`](https://huggingface.co/VikramPal/mistral-7b-instruct-v0.3-text2sql-bf16),
which is that adapter merged into the base model. Published separately so the fine-tune can be
re-merged, inspected, or stacked onto a differently-quantized base without downloading 13.5 GiB.

**This adapter is 335 611 085 B against the merge's 14 499 764 397 B, and it is the whole of the
difference between them.** Everything the fine-tune learned is here; nothing else was changed.

## What it was trained on

| | |
|---|---|
| Base | `mistralai/Mistral-7B-Instruct-v0.3` |
| Mixture | `gretelai/synthetic_text_to_sql` + `Salesforce/wikisql` + `b-mc2/sql-create-context`, 13 334 / 13 333 / 13 333 conversations |
| Kept | 39 531 of 40 000; 469 dropped for exceeding 2048 tokens |
| Loss on | completion only (`mask_mode: template`), 1 426 125 supervised tokens of 15 858 075 |
| Regime | LoRA r=32, alpha=64, dropout 0.05, on `q,k,v,o,gate,up,down` |
| Schedule | 2 epochs, lr 1e-4, effective batch 32, 2472 steps |
| Final train loss | 0.0540 |
| Wall clock | 3 h 49 m on one RTX PRO 6000 Blackwell |

Decontaminated against the eval split before training: 4 gretel, 16 wikisql and 3342
create-context conversations removed for overlapping an evaluation problem. `create-context`
is a training-only source -- it contributes to the adapter and is scored on nothing.

## What it scores

Evaluated on 2454 held-out problems drawn equally from gretel, wikisql and **spider**, 2-shot,
greedy, execution-free logic match:

**78.16% overall** (1918/2454), 0 unparseable, 0 truncated. By source:

| source | accuracy | in the training mixture? |
|---|---|---|
| wikisql | **93.89%** (768/818) | yes |
| gretel | **77.02%** (630/818) | yes |
| spider | **63.57%** (520/818) | **no** |

Spider is a third of the evaluation and none of the training mixture, so the 30-point gap
between it and wikisql is what this adapter does *not* transfer. 1144 of the 1918 correct
answers match the gold SQL as text; the other 774 are correct by execution equivalence.

## Using it

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3", dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(base, "VikramPal/mistral-7b-instruct-v0.3-text2sql-lora")
tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")
```

Call `model.merge_and_unload()` to get the bf16 checkpoint published above, byte-for-byte
modulo the merge's own float arithmetic.

## The rest of the campaign

This adapter is one artifact of a quantization study. The others:

| repo | what |
|---|---|
| [`...-text2sql-bf16`](https://huggingface.co/VikramPal/mistral-7b-instruct-v0.3-text2sql-bf16) | this adapter merged; the accuracy ceiling every quantized arm is measured against |
| [`...-text2sql-DynQuant-4bit`](https://huggingface.co/VikramPal/mistral-7b-instruct-v0.3-text2sql-DynQuant-4bit) | 3.96 GB, 78.08% |
| [`...-text2sql-DynQuant-3bit`](https://huggingface.co/VikramPal/mistral-7b-instruct-v0.3-text2sql-DynQuant-3bit) | 3.07 GB, 75.22% |

The signals DynQuant allocates from -- per-module activation saliency and gradient plasticity --
were harvested **during this LoRA run**, by forward and backward hooks on 226 modules, and
written alongside the adapter. That is the only reason the adapter and the quantization are the
same campaign: the fine-tune is where the allocation's inputs come from.

- Code: <https://github.com/kambojvikram/dynquant>
- `pip install dynquant`

## Limitations

Trained to emit a single SQL statement for a schema and a question, and nothing else. It is not
a general assistant any more, and the 2-epoch schedule at loss 0.054 is well into the regime
where it will answer off-task prompts in SQL. Outputs are not validated against a database and
have not been checked for injection-safe parameterization -- do not execute them against
anything you care about without review.