docs/research/README.md

# Research Notes

This repository started as a text-to-SQL fine-tuning research project and is now
being shaped into LocalSQL, a usable local/private SQL assistant.

## Core Result

The first strong result was Spider V1:

- Fine-tuned Qwen2.5-Coder-7B: **78.2%** result accuracy.
- Grok-4 baseline: 73.7%.
- DeepSeek-V3 baseline: 71.8%.

The method was frontier-disagreement DPO: run two strong models, execute their
SQL, and turn correctness disagreements into DPO preference pairs.

## BIRD Findings

BIRD is harder because schemas are larger, train/dev databases are disjoint, and
questions include external evidence. The important lessons so far:

- Clear correctness pairs helped the 7B model.
- Judge-resolved "both correct but stylistically different" pairs hurt BIRD,
  because BIRD rewards execution correctness, not SQL style.
- Reasoning distillation plus execution-based Best-of-N is the strongest story
  to package next, pending final artifact verification.

## Branches

- `main`: public product/reproducible baseline branch.
- `feat/localsql-product`: current cleanup branch for the LocalSQL product.
- `feat/bird-7b-clean`: GRPO/execution-reward research branch.
- `feat/excot-onpolicy`: reasoning-distillation and Best-of-N research branch.