Add LoRA adapter, model card and eval results

b04c6be verified 17 days ago

1.83 kB

base_model: meta-llama/Llama-3.1-8B-Instruct
library_name: peft
license: llama3.1
pipeline_tag: text-generation
tags:
  - lora
  - peft
  - text-to-sql
  - tinyforge
  - low-vram

tinyforge: Llama-3.1-8B-Instruct LoRA for text-to-SQL, trained on a 4 GB GPU

A 14 MB LoRA adapter (r=16) trained with tinyforge on a 4 GB RTX 3050 Ti laptop GPU using layer streaming: 200 steps in 9.5 minutes. Training data: 1,800 question/SQL pairs over six public seaborn tables (titanic, mpg, tips, car_crashes, planets, healthexp), each answer verified by executing the SQL. Use under the Llama 3.1 Community License.

Results (read the caveats)

100 questions about a table the model never saw (penguins), served with llama.cpp (adapter f16 on a Q4_K_M base), greedy decoding.

Model	Execution accuracy
This adapter	100%
Base told to "reply with exactly one SQLite query"	93%

Caveats, honestly:

100% is a ceiling: the instructed base is already at 93%, so this test cannot show how much headroom tuning adds. All 7 base misses are one malformed MAX/MIN call, so the gain is consistency, not new capability.
Questions use the same 12 templates as training, on a new schema: it measures transfer to new tables, not new kinds of questions.
On harder question shapes the same 8B model scored 98% vs 97% (no meaningful gain over a well-prompted base).
n=100, one run, no confidence interval. Raw numbers: eval_results.json.

Use

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "RajGMore/tinyforge-llama3.1-8b-sql-lora")