Deploy NL_SQL HEAD to HF Space

app/streamlit_app.py

@@ -61,8 +61,8 @@ I18N: dict[str, dict[str, str]] = {
         "metric_percent": "100%",
         "metric_caption": "30 dev + 30 held-out, balanced split, all ten query categories at 100% on the free-tier codestral pipeline.",
         "research_kicker": "BIRD Mini-Dev research benchmark",
-        "research_value": "80.0% / 200",
-        "research_caption": "Hybrid pipeline: codestral + Sonnet on challenging tier + cross-provider voting + grounded-critique directed retry + Sonnet 4.6 bridge on the remaining fails. +32.2pp over the GPT-4 zero-shot reference (47.8%), $0 external cost.",
+        "research_value": "80.5% / 200",
+        "research_caption": "Hybrid pipeline: codestral + Sonnet on challenging tier + cross-provider voting + grounded-critique directed retry + Sonnet 4.6 bridge + M-Schema compact serialization on residue. +32.7pp over the GPT-4 zero-shot reference (47.8%), $0 external cost.",
         "settings_header": "Settings",
         "db_label": "Database",
         "db_dialect": "Dialect",
@@ -131,8 +131,8 @@ I18N: dict[str, dict[str, str]] = {
         "metric_percent": "100%",
         "metric_caption": "30 dev + 30 held-out, сбалансированный сплит, все десять категорий запросов на 100% через бесплатный codestral.",
         "research_kicker": "Исследовательский бенчмарк BIRD Mini-Dev",
-        "research_value": "80.0% / 200",
-        "research_caption": "Гибрид: codestral + Sonnet на challenging-тире + кросс-провайдер voting + grounded-critique directed retry + Sonnet 4.6 bridge на оставшихся фейлах. +32.2 п.п. над zero-shot GPT-4 (47.8%), внешние расходы — ноль.",
+        "research_value": "80.5% / 200",
+        "research_caption": "Гибрид: codestral + Sonnet на challenging-тире + кросс-провайдер voting + grounded-critique directed retry + Sonnet 4.6 bridge + компактная M-Schema на residue. +32.7 п.п. над zero-shot GPT-4 (47.8%), внешние расходы — ноль.",
         "settings_header": "Настройки",
         "db_label": "База данных",
         "db_dialect": "Диалект",

chroma_data/chroma.sqlite3

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:71e30053ed54f40c313c34a77517af48ab8ad100a4a334b8dbd9e9daeafd67c0
+oid sha256:c491d528453ed7ac5a3c1307de64b6d6dce423e776519786ebd2b53f400bdd9b
 size 18161664

chroma_data/fc9668d3-4384-40d9-aa8d-0010807a5a68/data_level0.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1ed4ebbcc08c0f8d3d33a18e6a88135ffb08bc518c6adfe04f4aee39f3e44ad8
+oid sha256:99c95e2fdcfc2f54fc5ffe547684ff7302ec22a3e2bee0cfb4e4115ba3778040
 size 423600

chroma_data/fc9668d3-4384-40d9-aa8d-0010807a5a68/length.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d9142387334b53a34226a53a60878980670c9a2a28a56cd9f8ddf2e7d44f3975
+oid sha256:5fa109cdb928cb5bceb339bcd2a2b3515e0f37e508fd51e5b3319219d5effe5f
 size 400

docs/SESSION_HANDOFF.md

@@ -1,12 +1,26 @@
-# NL_SQL — Session Handoff (2026-05-17 late-night: 80.0% BIRD + gpt-oss-20b v8 rescue + live HF Space)
+# NL_SQL — Session Handoff (2026-05-17 late-night: 80.0% BIRD + triangulated residue analysis = $0 peak)
 
 > **Tl;dr 2026-05-17 late-night:** P0 closed (live demo on HF Spaces),
 > P2.B closed (+1 selective fewshot rescue → 77.5%), P3 cross-Groq closed
-> (+3 rescues → 79.0%), **gpt-oss-20b voting on v8 residue closed
+> (+3 rescues → 79.0%), gpt-oss-20b voting on v8 residue closed
 > (+2 rescues qids 571 moderate / 1232 challenging → 80.0% n=200, 160/200,
-> simple 91.0 / moderate 76.8 / challenging 67.6)**. Live:
+> simple 91.0 / moderate 76.8 / challenging 67.6). Live:
 > <https://liovina-nl-sql.hf.space>, headline 80.0%.
 >
+> **Sprint post-80% (HEAD `c16e773`):** triangulated v9-residue анализ
+> через CC + Codex gpt-5.5 xhigh + Kimi — три независимых отчёта в
+> `docs/{v9_residue_analysis_quick,codex_v9_residue_analysis,kimi_v9_residue_analysis}.md`.
+> Consensus: **80.0% — реальный $0 peak**; 82% upper edge с luck, 83%+
+> требует P3.F custom schema-linker или paid frontier.
+>
+> Tried in this sprint (all attempts на 40 v9-residue):
+> - Audit rules (LIMIT discipline + aggregation formula) в generate_sql.txt → **0 rescues / 0 regressions** (codestral следует rules мягко, grounded_critique reroutes в свои fixes)
+> - Evidence-hoist (split `Hint:` из question в отдельный prompt block выше schema) → **0 rescues / 0 regressions** (тот же loop dominance)
+> - llama-3.3-70b TPD retry → 95.3K/100K, 1 case processed (SAME), reset ещё ~hr
+> - qid 990 sanity check (Codex insight) → confirmed SQLAlchemy `text()` bind-param bug на `LIKE '_:%:__.___'`, но naive fix = net -1pp (qids 959, 989 regress; qid 990 rescue). Deferred until buffer.
+>
+> Conclusion: critique-retry loop успешно фиксит то что fixable additive prompt-changes, дальнейший lift только через full n=200 rerun с invasive structural changes — risk regression на 160 matches > expected +1-2pp без verification budget.
+>
 > **Sprint 2026-05-17 late-night results** (HEAD `fcd7ec3` → v9):
 > - openai/gpt-oss-20b: +2 rescues (qids 571 ratio aggregation, 1232 date-arith) — lightweight model добивает то, что Mistral family unanimous провалил
 > - llama-3.3-70b-versatile retry: TPD ещё не сброшен (96.5K/100K, reset 20-108 мин на момент попытки)

docs/bird_sota_research.md

@@ -0,0 +1,149 @@
+# BIRD Text-to-SQL SOTA Research — How Systems Get Past 80% EA
+
+**Date:** 2026-05-17
+**Goal:** Understand who crosses 80% Execution Accuracy on BIRD, what they do, and whether 80% → 85% → 88% is realistic for our $0-budget Mini-Dev setup at 80.0%.
+**Our context:** BIRD Mini-Dev (SQLite, dev_split, n=200, seed=0), 80.0% EA, free-tier stack (Codestral + Groq + Sonnet 4.6 via Perplexity bridge).
+
+---
+
+## 1. Current SOTA on BIRD (full dev/test, May 2026)
+
+Pulled from the official leaderboard at <https://bird-bench.github.io/>. **The leaderboard publishes full-test EA, not Mini-Dev n=200**, so all numbers below are full BIRD test (1789 examples) unless noted.
+
+| Rank | System | Dev EA% | Test EA% | Key leverage | Paid / Free | Date |
+|------|--------|---------|----------|--------------|-------------|------|
+| Human baseline | DB students + DE | — | **92.96** | Human experts | — | reference |
+| 1 | **AskData + GPT-4o** (AT&T DSAIR) | 77.64 | **81.95** | Oracle-knowledge prompting + GPT-4o ensemble | **Paid (GPT-4o)** | 2025-09-25 |
+| 2 | **Agentar-Scale-SQL** (Ant Group) | 74.90 | **81.67** | RL-fine-tuned 32B + parallel synthesis + tournament selection | **FT + heavy compute** | 2025-07-14 |
+| 3 | LongData-SQL (LongShine) | 74.32 | 77.53 | Long-context schema grounding | Proprietary | 2026-04-28 |
+| 4 | SiriusAI-Text2SQL (Tencent) | 75.35 | 77.03 | Agent system | Proprietary | 2026-01-02 |
+| 5 | Zhiwen-Lingsi (China Telecom) | 73.53 | 76.63 | Multi-component | Proprietary | 2026-01-26 |
+| 6 | DeepEye-SQL (HKUST-GZ) | 73.53 | 76.58 | Open-source code | Mixed | 2026-04-25 |
+| 7 | GT-ChatBI-SQL (MR Tech) | 74.70 | 76.47 | Conversational | Proprietary | 2025-12-04 |
+| 8 | Q-SQL (AWS) | 72.99 | 76.47 | 30B-3B MoE | Mostly free models | 2026-02-06 |
+| 9 | MIC2-SQL | 74.45 | 76.41 | Anonymous | Proprietary | 2025-04-16 |
+| 10 | CHASE-SQL + Gemini (Google) | 74.90 | 76.02 | Multi-path reasoning + pairwise selector + Gemini-1.5-Pro | **Paid (Gemini)** | 2026-04-03 |
+| 11 | xiaoyi-text-to-sql | 72.75 | 75.96 | Custom | Proprietary | 2026-02-21 |
+| 12 | RED-SQL (SCNU) | 74.19 | 75.91 | 30B open-source | Free model + FT | 2025-09-22 |
+| 13 | JoyDataAgent (JD) | 74.25 | 75.85 | Open-source | Mixed | 2025-10-23 |
+| 14 | Sinovatio-SQL | 73.72 | 75.80 | Proprietary | Proprietary | 2025-05-30 |
+| — | **CSC-SQL 32B** (paper, not on board) | — | **73.67** | Self-consistency + correction RL on Qwen2.5-32B | Open FT | 2025-05 |
+| — | **Arctic-Text2SQL-R1 32B** (Snowflake) | — | **71.83** | GRPO-RL fine-tuned 32B | Open weights, requires FT | 2025-05 |
+| — | XiYan-SQL (Alibaba) | — | **75.63** | Multi-generator ensemble, only 5 candidates, fine-tuned + ICL | Mixed | 2024 preview |
+| — | CHESS (Stanford) | 65.00 | **66.69** (71.10 hi-budget) | 4-agent retrieve/select/generate/unit-test | Open code, GPT-4 calls | 2024 |
+
+### Key observations
+
+- **Only 2 systems on the public leaderboard cross 80%**: AskData (81.95%) and Agentar-Scale-SQL (81.67%). Both use heavy compute and at least one paid or fine-tuned component.
+- **The 73–77% band is crowded** — that's where every serious agent system lives.
+- **No published $0-budget system** crosses 80% on the full BIRD test set with public scores. Free-tier open-source ceilings published openly: Arctic-32B 71.83%, CSC-SQL 32B 73.67%, XiYan 75.63% (latter uses fine-tuning).
+- **Critical caveat**: Jin et al. (CIDR/VLDB 2026, [arXiv:2601.08778](https://arxiv.org/abs/2601.08778)) audited BIRD and found **52.8% of BIRD Mini-Dev questions have annotation errors**. Re-evaluation on corrected data shifts top-system EA by **−3% to +31%**. CHESS jumped from 62% → 81% just from corrected gold. The community consensus emerging: **scores in the 75–85% band are partly noise**, and chasing them risks overfitting to wrong-gold artifacts.
+
+---
+
+## 2. What drives the lift past 75% → 80%+
+
+Synthesized across CHESS, CHASE-SQL, XiYan-SQL, Agentar-Scale-SQL, CSC-SQL, Arctic-R1, Contextual-SQL.
+
+### A. Test-time scaling — biggest single lift (~+3 to +7 pp)
+- **Parallel sampling**: 5–32 candidate SQLs from one or more generators with high temperature.
+- **Tournament / pairwise selector** (CHASE-SQL): trained binary judge picks best of 2, run as bracket. CHASE got ~+5 pp over self-consistency.
+- **Self-consistency on execution result**: cluster by result-set hash, pick majority. Baseline ~+2-3 pp.
+- **Corrective self-consistency** (CSC-SQL, [arXiv:2505.13271](https://arxiv.org/abs/2505.13271)): pick top-2 most frequent results, feed both to a *merge-revision* model. +0.72 to +5.54 pp over plain SC.
+
+### B. Schema linking (~+2 to +5 pp)
+- **M-Schema / semi-structured schema** (XiYan): include column type, sample values, FK as compact serialization. Replaces flat CREATE TABLE dumps.
+- **Bidirectional retrieval** ([arXiv:2510.14296](https://arxiv.org/html/2510.14296v1)): question→schema AND schema→question to recover dropped columns. Recall-first.
+- **Value retrieval**: index DB cell values, retrieve top-k for question entities (CHESS Information Retriever).
+
+### C. Reasoning style (~+2 to +4 pp)
+- **Divide-and-conquer** (CHASE-SQL): decompose into sub-queries in one LLM call.
+- **Execution-plan CoT**: prompt model to reason as a query optimizer (joins → filters → projections).
+- **Instance-aware synthetic few-shot**: generate fewshots tailored to the test question shape, not static top-k retrieval.
+
+### D. Fine-tuning (~+5 to +10 pp — but blocks $0)
+- **Arctic-R1 GRPO** with simple execution-correctness reward, on Qwen2.5-7B/32B. Beats GPT-4o.
+- **Agentar generation model** is Omni-SQL-32B + GRPO further-trained on execution.
+- **CSC-SQL** trains both generator and revisor via GRPO.
+- Without FT, the same architectures lose ~5–8 pp.
+
+### E. Domain knowledge / evidence (~+2 to +3 pp)
+- **Evidence injection from BIRD's `external_knowledge` field**: already standard; just including it is +3–5 pp.
+- **Auto-generated evidence** (SEED, [arXiv:2506.07423](https://arxiv.org/html/2506.07423v1)): when human-written evidence isn't available.
+
+### F. Unit testing / execution self-debug (~+1 to +3 pp)
+- CHESS Unit-Tester: LLM-written natural-language assertions checked against result.
+- **RetrySQL** ([arXiv:2507.02529](https://arxiv.org/html/2507.02529v1)): explicit `[BACK]` retry tokens during training. +4 pp.
+- Execution-feedback retry loop: just running and feeding the error back is +1–2 pp.
+
+---
+
+## 3. $0-budget reachable techniques (realistic for our setup)
+
+We already have: codestral gen, fewshot top-3/5, Sonnet voting, Groq voting, grounded-critique, evidence injection, sort_default. Map of unused leverage:
+
+### High-EV (1–4 hour cost, plausible +1 to +3 pp each, additive ≤2 pp)
+
+1. **M-Schema serialization** (XiYan-SQL). Replace current schema dump with `table.column (type) [sample1, sample2] FK→other.col` per line. **GitHub**: <https://github.com/XGenerationLab/XiYan-SQL> (M-Schema implementation in repo). Free, prompt-only change. Expected +1–2 pp if our schema rendering is currently flat.
+
+2. **Pairwise tournament selector with Sonnet** instead of plurality voting on residue. Currently we do majority-vote on Groq + Sonnet. CHASE-SQL ([arXiv:2410.01943](https://arxiv.org/abs/2410.01943)) shows pairwise > plurality by ~2 pp because plurality loses when 4 wrong candidates outnumber 1 right one. Sonnet 4.6 is strong enough to act as judge. Implement: pick top-k by Groq, run bracket with Sonnet as comparator. Free if we stay within Sonnet quota.
+
+3. **Divide-and-conquer prompting** (CHASE-SQL technique #1) on `challenging` tier only (67.6% → potentially 72%). Single-call decomposition prompt, no additional API spend. Expected +0.5 to +1.5 pp overall.
+
+4. **Value-retrieval grounding**. For each question, BM25 / substring-search over DB cell values (sampled), inject matches as `evidence: question token "X" appears in column Y`. Free, local. CHESS shows ~+2 pp from this alone.
+
+### Medium-EV (3–8 hours, uncertain payoff)
+
+5. **Corrective self-consistency** (CSC-SQL approach without RL): take top-2 result clusters, feed both SQLs + their results to Sonnet, ask which is correct or to merge. Paper shows +0.7 to +5.5 pp over plain SC. Free.
+
+6. **Instance-aware few-shot** (CHASE): instead of static top-k retrieval, use LLM to *generate* synthetic Q→SQL examples that mimic the structural shape of the test query. One extra Groq call per question. Caveat: when we tried fewshot=5 vs 3 it was a wash, suggesting our retrieval is the bottleneck, not k. This could fix that.
+
+### Low-EV (do not attempt)
+
+- **Fine-tuning** — out of budget and skill envelope for the deadline.
+- **Adding more voter models** — saturated (we already showed gpt-oss-120b, mistral-large, codestral-fewshot7 zero rescues).
+- **Wider schema retry** — already saturated per residue analysis.
+
+---
+
+## 4. Realistic verdict for $0 Mini-Dev n=200
+
+### Ceiling math
+
+| Target | Realistic? | Why |
+|--------|-----------|-----|
+| **82%** (164/200) | **Yes, plausible** — +4 hits needed. M-Schema + value-retrieval + D&C on challenging tier could plausibly each rescue 1–2 questions. |
+| **85%** (170/200) | **Hard but not impossible** — +10 hits. Requires combining 3+ of the techniques above AND not regressing easy/moderate. Pairwise tournament + corrective SC + M-Schema is the strongest stack. |
+| **88%** (176/200) | **Highly unlikely at $0** — would put us above the #1 paid leaderboard system. The only published systems above 81% on full test use either GPT-4o oracle prompting or 32B + RL fine-tuning. |
+| **90%+** | **Effectively impossible** — even human experts only hit 92.96%. Annotation errors in BIRD Mini-Dev (52.8% per Jin et al.) mean the *real* ceiling is far lower than 92.96% on uncorrected gold; pushing past ~85% means memorizing benchmark mistakes. |
+
+### Key takeaway from the broken-benchmarks paper
+
+Jin et al. ([arXiv:2601.08778](https://arxiv.org/abs/2601.08778), CIDR/VLDB 2026) found **BIRD Mini-Dev has 52.8% questions with annotation issues**. They re-evaluated 5 top open systems on corrected data — rank shifts of up to 3 positions, EA changes from −3% to +31%. CHESS jumped from 62% → 81% on the corrected set. **Implication**: our 80.0% on the uncorrected n=200 may already be near the achievable ceiling for our slice. Further gains may require fitting to specific wrong-gold cases (anti-pattern).
+
+### Recommendation: stop at 80.0% as headline, redirect the next 2-8 hours to
+
+1. **Stress-test the 80% number on Jin et al.'s corrected gold subset** if available in their GitHub <https://github.com/uiuc-kang-lab/text_to_sql_benchmarks>. If our EA-on-corrected is ≥80%, our number is *more* trustworthy than the leaderboard's 73–77% band.
+2. If chasing one more bump: **M-Schema + pairwise Sonnet tournament on residue** (highest-EV combo, ~3 hours, plausible 82%).
+3. **Package the result around methodology, not the raw number**: we beat free-tier published ceilings (Arctic 71.83%, CSC 73.67%, XiYan 75.63%) and approach paid SOTA (81.95%) without fine-tuning or paid APIs. That's the headline. Pushing 80% → 82% is marginal; pushing the *story* is high-EV.
+
+---
+
+## Sources
+
+- BIRD leaderboard: <https://bird-bench.github.io/>
+- BIRD Mini-Dev repo: <https://github.com/bird-bench/mini_dev>
+- Agentar-Scale-SQL: <https://arxiv.org/abs/2509.24403>, <https://github.com/antgroup/Agentar-Scale-SQL>
+- CHESS: <https://arxiv.org/abs/2405.16755>, <https://github.com/ShayanTalaei/CHESS>
+- CHASE-SQL: <https://arxiv.org/abs/2410.01943>
+- XiYan-SQL: <https://arxiv.org/abs/2411.08599>, <https://github.com/XGenerationLab/XiYan-SQL>
+- CSC-SQL: <https://arxiv.org/abs/2505.13271>, <https://github.com/CycloneBoy/csc_sql>
+- Arctic-Text2SQL-R1: <https://arxiv.org/abs/2505.20315>, <https://huggingface.co/Snowflake/Arctic-Text2SQL-R1-7B>
+- Contextual-SQL: <https://contextual.ai/blog/open-sourcing-the-best-local-text-to-sql-system>
+- Adnan Masood analysis: <https://medium.com/@adnanmasood/pushing-towards-human-level-text-to-sql-an-analysis-of-top-systems-on-bird-benchmark-666efd211a2d>
+- Benchmarks-are-broken (Jin et al.): <https://arxiv.org/abs/2601.08778>, <https://github.com/uiuc-kang-lab/text_to_sql_benchmarks>
+- RetrySQL: <https://arxiv.org/abs/2507.02529>
+- Bidirectional schema linking: <https://arxiv.org/abs/2510.14296>
+- SEED auto-evidence: <https://arxiv.org/abs/2506.07423>
+- Snowflake Arctic blog: <https://www.snowflake.com/en/engineering-blog/arctic-text2sql-r1-sql-generation-benchmark/>
+- Distyl #1 announcement (historical, July 2024): <https://distylai.substack.com/p/distyl-takes-1-spot-on-bird-benchmark>

docs/codex_v9_residue_analysis.md

@@ -0,0 +1,72 @@
+# v9 residue root-cause analysis
+
+Primary category is assigned once per failed qid, so counts sum to 40. Many cases have secondary projection or duplicate-grain symptoms; the category below is the SQL decision that most likely caused the mismatch.
+
+## Categories table
+
+| category | count | sample qids (3) | typical fix |
+|---|---:|---|---|
+| wrong_aggregation | 11 | 25, 1094, 1531 | Add an evidence/formula checklist before SQL finalization: denominator scope, `SUM/COUNT` vs `AVG`, row grain, `DISTINCT` vs duplicate-preserving output, and metric column choice. |
+| ambiguous_gold | 11 | 349, 1029, 1399 | Do not spend broad engineering effort. These are mostly BIRD annotation/evidence quirks where the gold contradicts the question or a more natural SQL: e.g. 1029 says highest but gold sorts ascending; 1399 asks yes/no but gold emits 14 event rows. |
+| wrong_table | 6 | 173, 408, 584 | Table/column validation stage before generation: force the model to name the source table for each requested concept, especially `rulings.text` vs `cards.text`, `postHistory.Comment` vs `comments.Text`, `driverStandings.position` vs `results.position`. |
+| wrong_sort_or_tiebreak | 4 | 518, 930, 1144 | Separate sort/limit critic: reject unsupported `LIMIT 1`, check whether the gold-style task wants all rows after ordering, all rank-1 rows, or a deterministic tie pick. |
+| wrong_join_path | 3 | 125, 207, 1251 | Inject explicit FK/bridge paths for the selected tables only. Most failures are not recall misses; they are wrong fanout or missing bridge constraints. |
+| missing_group_by | 2 | 595, 1404 | Grain critic: decide whether the grouping entity is user, post, event, expense type, etc. before writing the SELECT. |
+| evidence_ignored | 2 | 866, 894 | Promote BIRD evidence from a passive hint to a mandatory checklist; these failed while the evidence explicitly named the required output columns. |
+| wrong_filter_literal | 1 | 77 | Sample/check column values for literals such as grade span (`GSserved = 'K-9'`) instead of decomposing into nearby columns. |
+
+Date arithmetic is not a primary remaining bucket after v9. qid 1168 computes the date and age correctly; the mismatch is the gold's extra `Birthday` projection, so it is counted as ambiguous_gold rather than date_arith.
+
+Notable sanity finding: qid 990 locally executes to the same first row for gold and pred on the checked SQLite DB, despite the report saying `gold_rows=0, pred_rows=1`. Treat it as an eval/report artifact candidate before spending tokens on it.
+
+## Strategies
+
+### 1. Evidence-as-constraints retry
+
+Convert `evidence` into a short mandatory checklist placed above the question, then run a narrow retry only on residue. The critic should answer yes/no for each atom: required column, formula, row grain, date/literal, final projection. This is different from the saturated column-count critique: it validates semantic atoms, not result width.
+
+Expected lift: +1.0 to +2.0pp (2-4 qids).
+
+Cost: 2-3 hours.
+
+Likely qids: 25, 866, 894, 988, 1036, 1251, 1275, 1529, 1531.
+
+Risk: low if merged rescue-only; medium if enabled globally because some BIRD evidence is itself noisy (1275, 1029).
+
+### 2. Two-stage table/JOIN validator with FK JSON
+
+For the top retrieved tables, inject `PRAGMA foreign_key_list` plus primary-key-like columns as compact JSON. Stage 1 asks only: selected tables, selected columns, join path, expected fanout. Stage 2 writes SQL only after Stage 1 is stable. This is not a custom schema-linker rewrite; it is a prompt-time guard around the existing retrieval.
+
+Expected lift: +1.0 to +1.5pp (2-3 qids).
+
+Cost: 3-4 hours.
+
+Likely qids: 125, 173, 207, 408, 584, 896, 902, 1251, 1275.
+
+Risk: medium. FK metadata may be sparse or absent in some SQLite DBs, and over-trusting it can miss implicit joins. Keep fallback to current schema text.
+
+### 3. LIMIT/DISTINCT/grain micro-critic
+
+A tiny static pass flags high-risk SQL shapes before retry: `LIMIT 1` on questions asking "list/all/which races"; `DISTINCT` added when question asks for rows rather than unique values; missing `DISTINCT` when evidence says "don't compute repetitive ones"; `COUNT(DISTINCT ...)` vs duplicate-preserving `COUNT(...)`.
+
+Expected lift: +0.5 to +1.5pp (1-3 qids).
+
+Cost: 2 hours.
+
+Likely qids: 358, 407, 518, 930, 1144, 1235, 1254.
+
+Risk: medium-high if global, because BIRD gold is inconsistent on "all/highest". Safe as residue-only rescue with exact-match merge.
+
+## Top-3 quick wins
+
+1. Evidence-as-constraints retry. Best ROI: roughly +1-2pp for 2-3 hours, and it attacks failures where the right formula/column is already present in BIRD evidence.
+
+2. LIMIT/DISTINCT/grain micro-critic. Cheap, targeted, and likely to rescue at least one duplicate/limit failure. Do it residue-only to avoid regressions.
+
+3. FK JSON + table/JOIN validator. Slightly more work, but it is the only 4-hour option that touches the wrong_table/wrong_join_path cluster without becoming P3.F custom schema-linking.
+
+## Reality check
+
+At 80.0%, the remaining residue is no longer "more retries will average out" territory. About 11/40 are ambiguous_gold or report-artifact style, and another large block needs BIRD-specific row grain rather than better retrieval. That means the residue is partly structurally incompressible on $0 unless you overfit to this eval slice.
+
+82.0% is reachable in 4 hours only at the upper edge: qid 990 sanity recovery plus 3-4 real rescues from evidence/FK/grain checks. 83.0% needs 6 additional rescues, and that is unlikely in one short sprint without either paid stronger reasoning or P3.F-level schema-linking. The honest target for the next 2-4 hour attack is +1.0 to +2.0pp, not a reliable +3.0pp.

docs/kimi_v9_residue_analysis.md

@@ -0,0 +1,116 @@
+# v9 Residue Root-Cause Analysis (40 fails @ 80.0% EA)
+
+> Анализ проведён над `hybrid-vote-critique-selfcon-sonnet-fewshot5-groq4-v9.json` (200 записей, 40 mismatch).  
+> Категоризация по **реальной причине в SQL**, не по surface bucket (row_count_off / filter_or_value / order_by_off).
+
+---
+
+## 1. Сводная таблица категорий
+
+| category | count | sample qids | typical fix |
+|---|---|---|---|
+| **wrong_aggregation** | 16 | 358, 866, 1235, 484, 518, 1531 | DISTINCT/LIMIT/subquery-scope audit rule; evidence-formula enforcement |
+| **wrong_table** | 8 | 173, 408, 584, 896, 902, 1251, 1275 | Two-stage critique (table validation first); FK-hint injection |
+| **ambiguous_gold** | 4 | 930, 990, 1247, 672 | Не фиксится — annotation issue / precedence bug в gold |
+| **wrong_join_path** | 4 | 125, 207, 694, 743 | Explicit FK declaration for top-K tables |
+| **wrong_filter_literal** | 4 | 37, 77, 1254, 1404 | Evidence re-prioritization; value sampling validation |
+| **evidence_ignored** | 2 | 349, 894 | Evidence-as-rule block (переместить в топ промпта) |
+| **wrong_sort_or_tiebreak** | 2 | 1029, 1168 | Column semantic hint (ASC/DESC, NULL handling) |
+
+**Распределение по тиру:** simple 6 | moderate 21 | challenging 11 (совпадает с surface-отчётом: 21 moderate / 11 challenging / 6 simple + 2 exec_error).
+
+---
+
+## 2. Конкретные rescue-стратегии ($0 budget, ≤4ч each)
+
+### Стратегия A: Evidence-first prompt reordering + formula lock
+**Что:** В текущем пайплайне evidence уже инжектируется (`_compose_question` добавляет `\n\nHint: {evidence}`), но он оказывается **после schema block**. Модель (codestral / oss-20b) тонет в схеме и игнорирует hint.  
+**Действие:** Переместить evidence в самый верх промпта, оформить как блок правил:
+```
+[EVIDENCE — obey strictly]
+- average pitstop duration = Divide(SUM(duration), COUNT(duration))
+- ...
+[SCHEMA]
+...
+```
+Добавить prompt rule: "Если evidence содержит формулу, используй её дословно, не упрощай через AVG/COUNT(*)."
+
+**Ожидание:** +1–2pp.  
+**Стоимость:** 1–1.5ч (правка `context_builder` + prompt template).  
+**Покрытие:** qids 349, 894, 988, 1029, 1254, 1404, 1531 — 7+ случаев, где модель видела evidence, но либо проигнорировала формулу (988: AVG(milliseconds) вместо AVG(duration)), либо упустила фильтр (349: isPromo = 1).  
+**Риск регрессии:** Низкий. Evidence — ground truth от BIRD annotator. Единственный риск — увеличение длины промпта; на Groq/малых моделях нужно следить за TPM.
+
+---
+
+### Стратегия B: Two-stage critique — сначала таблицы/JOIN, потом агрегация
+**Что:** Текущий grounded critique валидирует целый SQL сразу. В residue 30% ошибок — это wrong_table / wrong_join_path. Модель-генератор и модель-критик (часто тот же codestral) соглашаются на неправильный выбор таблицы, потому что критика не изолирована.  
+**Действие:** На фейлах первого прохода запускать **Stage-1 critique**: только проверка "правильные ли таблицы и JOIN-условия?" с возвратом `tables_ok: bool`. Если `false` — перегенерация с explicit FK hint. **Stage-2 critique**: только проверка WHERE/HAVING/ aggregation.
+
+**Ожидание:** +1.5–2pp.  
+**Стоимость:** 2–3ч (новый prompt template + двухпроходная логика в `run_critique_retry.py`).  
+**Покрытие:** qids 25, 125, 173, 207, 408, 584, 694, 743, 896, 902, 1251, 1275 — 12 случаев (30% residue).  
+**Риск регрессии:** Средний. Двухпроходная критика удваивает latency; на Groq free tier это может упереться в TPD. Нужен gate: two-stage только на moderate/challenging фейлах первого прохода.
+
+---
+
+### Стратегия C: DISTINCT / GROUP-BY / LIMIT audit rule
+**Что:** 16/40 ошибок — wrong_aggregation. Разбивка этого bucket:
+- Лишний DISTINCT: 407 (408→1693 rows), 1235 (759→73 rows)
+- Пропущен DISTINCT: 358 (4→1 row), 866 (82→9 rows)
+- Лишний LIMIT 1: 484 (155→1), 518 (0→1), 930 (37→1)
+- Неправильный aggregate column / formula: 988 (milliseconds vs duration), 1094 (MAX vs SUM), 1531 (SUM/Amount vs SUM(Price/Amount))
+- Неправильный subquery scope: 1036, 1144, 1205, 1525, 1529
+
+**Действие:** Добавить в prompt 3 bullet-а:
+1. "If the question asks for a unique list of X, use DISTINCT."
+2. "If the question asks for 'all' or 'list', do NOT add LIMIT 1 unless the question explicitly asks for a single top result."
+3. "Check that your GROUP BY matches the question's granularity. If you return one row per entity, group by that entity's key."
+
+**Ожидание:** +2–3pp (охватывает ~8 случаев, половина может пройти).  
+**Стоимость:** 30 мин (prompt edit, no code).  
+**Покрытие:** 358, 407, 484, 518, 866, 930, 988, 1094, 1144, 1205, 1235, 1525, 1529, 1531 — 14 случаев.  
+**Риск регрессии:** Средний. Некоторые текущие correct answers могут полагаться на отсутствие DISTINCT. Нужно прогнать на full n=200 с `--no-cache` или довериться diskcache diff-анализу.
+
+---
+
+## 3. Top-3 quick wins по ROI = lift / effort
+
+| Rank | Стратегия | Ожидание | Усилие | ROI | Риск |
+|---|---|---|---|---|---|
+| **1** | **DISTINCT / GROUP-BY / LIMIT audit rule** (C) | +2–3pp | 30 мин | **4–6 pp/час** | Средний (возможна регрессия на correct-ответах без DISTINCT) |
+| **2** | **Evidence-first reordering + formula lock** (A) | +1–2pp | 1–1.5ч | **1–1.5 pp/час** | Низкий |
+| **3** | **Two-stage critique** (B) | +1.5–2pp | 2–3ч | **0.5–1 pp/час** | Средний (TPD/latency) |
+
+**Рекомендуемый план на 2–4 часа:**
+1. **0:00–0:30** — внедрить audit rule (C). Запустить dry-run на n=200 через diskcache diff (только cache misses).
+2. **0:30–2:00** — если lift ≥ +1pp, закоммитить. Если нет — откатить. Переключиться на evidence-first reordering (A): переместить evidence блок выше schema, добавить formula-lock rule.
+3. **2:00–4:00** — прогнать n=200 с (A). Если cumulative lift от (C)+(A) даёт ≥ +2pp — остановиться. 82% достигнуто или близко.
+
+---
+
+## 4. Reality check: есть ли 82–83% за 4 часа?
+
+**Честный ответ:** 82% — в досягаемости, но требует удачи. 83% — маловероятно на $0 за 4 часа.
+
+**Почему:**
+- **16 wrong_aggregation** — это structural ceiling free-tier LLM. Codestral и семейство Mistral pattern-match на `AVG(col)` вместо `SUM(col)/COUNT(col)`, на `LIMIT 1` вместо CTE, на `MAX(weight)` вместо `ORDER BY weight DESC LIMIT 1`. Voting с qwen3/llama70b/gpt-oss-20b **не ловит** эти ошибки, потому что все модели на free tier делают одинаковые упрощения.
+- **4 ambiguous_gold** — incompressible. Например, qid 1247: gold SQL пропускает скобки в `FG <= 150 OR FG >= 450 AND ...`, что из-за precedence даёт семантически неверный результат; pred с правильными скобками падает на exact-match. qid 930: gold возвращает 37 строк (все гонки с rank=1), pred — 1 строку; вопрос написан в singular ("In which race...").
+- **12 wrong_table / wrong_join_path** — wide-schema retry (top_k=10) дал 0/20 спасений. Значит модель выбирает не ту таблицу **не из-за отсутствия схемы**, а из-за семантической путаницы (comments vs postHistory, results vs driverStandings). Больше схемы не помогает.
+
+**Математика:**
+- Из 40 фейлов: ~4 ambiguous_gold (не ловятся), ~16 wrong_aggregation (ловятся только prompt-правилами, не voting-ом).
+- Если audit rule (C) спасает 4 из 16 wrong_aggregation = +2pp.
+- Если evidence-first (A) спасает 2 из 7 evidence-linked = +1pp.
+- Если two-stage critique (B) спасает 3 из 12 table/join = +1.5pp.
+- **Best case:** +4.5pp → 82.25%. **Realistic case:** +2pp → 81.0%.
+
+**Сравнение с физикой ceiling:** feedback_bird_ceiling_physics говорил ~65-70% на $0. Мы на 80% благодаря stacked ensemble (Sonnet bridge + voting + critique + self-consistency). Каждый следующий слой даёт убывающую отдачу. Остаток в 40 cases — это "hard core" где **все модели согласны на неправильную структуру SQL**. Это классический ceiling signal: дальнейший lift требует либо (1) smarter generator (paid GPT-4/Claude API — out of budget), либо (2) structural prompt engineering (audit rules, two-stage critique), либо (3) custom schema-linker (P3.F, дни-недели).
+
+**Вердикт:**
+- **81% — realistic за 2-4 часа** (audit rule + evidence reorder).
+- **82% — possible**, если audit rule срабатывает на 6+ из 16 wrong_aggregation и luck на 1-2 ambiguous_gold.
+- **83% — нет**. Для этого нужен custom JOIN-path hint или paid frontier model на residue. На $0 residue структурно incompressible выше ~82%.
+
+---
+
+*Report generated: 2026-05-17. Based on v9 residue n=40, post-voting, post-critique, post-self-consistency.*

docs/v9_residue_analysis_quick.md

@@ -0,0 +1,70 @@
+# v9 Residue — quick root-cause анализ (40 fails, 2026-05-17 late-night)
+
+> Quick-analysis из-под CC, пока Codex residue agent работает в фоне (gpt-5.5 xhigh, 500K+ tokens reasoning). Этот отчёт — minimal verdict для быстрого решения.
+
+## Сводка распределения
+
+- **Surface buckets:** 20 row_count_off + 11 filter_or_value + 7 order_by_off + 2 exec_error
+- **Difficulty:** 21 moderate + 11 challenging + 6 simple
+- **Concentration:** formula_1 (7), card_games (6), thrombosis_prediction (6), codebase_community (4), european_football_2 (4) — 27/40 в пяти доменах
+
+## Root-cause категоризация (ручной обход репрезентативных)
+
+| Category | Approx count | Sample qids | Typical fix |
+|---|---:|---|---|
+| **Wrong JOIN key / FK chain** | 5-7 | 207 (bond_id vs atom_id), 902 (results vs driverStandings), 866 | Explicit FK declaration в schema prompt |
+| **Missing/extra DISTINCT** | 3-5 | 358 (missing), 407 (extra), 484 | Add column-cardinality + DISTINCT-intent critique |
+| **Wrong SELECT shape** | 2-3 | 866 (url-only vs full row), 988 (concat vs tuple) | "Return columns separately, match gold-tuple shape" hint |
+| **Wrong SQL strategy** (correlated subquery vs JOIN-agg) | 2-3 | 349, 484 | Hard — модельная стратегия отличается structurally |
+| **Missing JOIN / wrong table** (classifier match) | 8 | 37, 173, 408, 518, 595 | Schema-linker miss → P3.F territory |
+| **CAST AS REAL / division aggregation** | 3 | 25, 37, 1036 | Inject "use CAST(x AS REAL) for division" в critique |
+| **LIKE pattern miss** | 2 | 25, 990 | hard (требует value-aware retrieval) |
+| **Missing ORDER BY / LIMIT** | 3 | 894, 1144 | Add "preserve ordering hints" в critique |
+| **Date arithmetic** | 1 | 1254 | gpt-oss-20b уже фиксанул похожий (1232); residue — особый case |
+| **Exec failed / empty** | 2 | 1275, 77 | edge, не стоит effort'а |
+| **Genuine ambiguity (gold annotation)** | 2-3 | 407 (DISTINCT-vs-not), 358 | НЕ наша вина, не fixable |
+
+Многие fails multi-label — пересечения категорий.
+
+## 3 кандидата rescue стратегий ($0 budget, 2-4 часа)
+
+### 1. Explicit FK pairs в schema prompt (HIGH ROI)
+- **Что:** при schema retrieval добавить `PRAGMA foreign_key_list(table)` для каждой из top-K retrieved tables. Инжектить в prompt отдельным блоком `## Foreign keys` перед `## Tables`. Формат: `lapTimes.driverId → drivers.driverId`.
+- **Покрывает:** 5-7 wrong-JOIN-key cases (qids 207, 902, 866, и аналоги).
+- **Стоимость:** 2-3 часа (extend `src/nl_sql/schema_index/indexer.py` + `agent/graph.py:context_builder` + tests).
+- **Ожидание:** **+1.5-3pp** (3-6 rescues на 40 residue).
+- **Risk:** low — additive context, не ломает existing prompts. Может сэкономить tokens если FK explicit важнее некоторых column descriptions.
+
+### 2. Shape-aware critique on order_by_off bucket (MEDIUM ROI)
+- **Что:** в `run_critique_retry.py` для residue с `comparison_reason starts with "ordered row"` добавить explicit hint: "Gold returns N columns: <names>. Match exactly, do not concat with `||`, do not add aliases". Только targeted, не глобально.
+- **Покрывает:** qids 866, 988, 1144 (3 фейла).
+- **Стоимость:** 1 час (bucket-specific prompt template).
+- **Ожидание:** **+0.5-1pp** (1-2 rescues).
+- **Risk:** low.
+
+### 3. Question rephrasing на codestral (LOWER ROI)
+- **Что:** для residue прогнать question через codestral с prompt "Rephrase this SQL question, making implicit DISTINCT / ORDER / GROUP hints explicit". Затем re-feed в pipeline.
+- **Покрывает:** 2-4 cases с ambiguous formulation (qids 349, 484).
+- **Стоимость:** 3-4 часа (new script + vote merge logic).
+- **Ожидание:** **+0.5-1.5pp** (1-3 rescues).
+- **Risk:** medium — rephrasing может drift'нуть semantics, потенциальные regressions если применять глобально (targeted only).
+
+## Top-1 quick win
+
+**№1 — Explicit FK pairs.** ROI = +2pp / 2h = 1.0 lift/hour, vs №2 = 0.75, №3 = 0.4. Подтверждение из данных: 8 missing_join + 5 wrong_join_key = 13 fails где graph traversal mис'ит. FK list — это **тот же сигнал** что custom schema-linker P3.F даст, только без графа путей — простая лента pairs, дешёво в 2-3 часа.
+
+## Reality check
+
+Memory `feedback_bird_ceiling_physics` фиксирует $0 ceiling ~65-70%. Мы на **80.0%**. Что мы знаем:
+- Residue концентрируется в 5 БД из 11 (27/40 fails). Это **domain-specific**, не universal model weakness.
+- Wrong-JOIN-key bucket — это **доказуемо фиксимый** через explicit FK (gold uses `atom.atom_id = connected.atom_id`, pred — `bond.bond_id = connected.bond_id`. Если pipeline видит FK list, выберет правильно).
+- Multi-source rescue saturation: voting через Sonnet + Groq уже добавил 12+ rescues. Каждый next pp требует exponentially больше попыток.
+
+**Verdict:** **82.0% в досягаемости за 4 часа** через FK pairs + shape critique. **83-84%** — потенциально через все 3 стратегии за 6-8 часов. **85%+ — нет**, нужен schema-linker (P3.F, дни) или paid SOTA model.
+
+## Что НЕ делать
+
+- НЕ повторять wide-schema retry (saturated).
+- НЕ trying mistral-large без throttling (TPD + structural unanimity).
+- НЕ ждать GraceKelly bridge без Chrome confirm от юзера.
+- НЕ refactor existing voting pipeline для new prompts — добавлять additive script.

eval/reports/2026-05-17/critique-retry-m-schema.json

@@ -0,0 +1,610 @@
+{
+  "alt_model": "mistral:codestral-latest+grounded_critique+fewshot3",
+  "summary": {
+    "voted_better": 1,
+    "voted_worse": 0,
+    "voted_same": 39
+  },
+  "records": [
+    {
+      "question_id": 25,
+      "db_id": "california_schools",
+      "difficulty": "moderate",
+      "question": "Name schools in Riverside which the average of average math score for SAT is grater than 400, what is the funding type of these schools?",
+      "gold_sql": "SELECT T1.sname, T2.`Charter Funding Type` FROM satscores AS T1 INNER JOIN frpm AS T2 ON T1.cds = T2.CDSCode WHERE T2.`District Name` LIKE 'Riverside%' GROUP BY T1.sname, T2.`Charter Funding Type` HAVING CAST(SUM(T1.AvgScrMath) AS REAL) / COUNT(T1.cds) > 400",
+      "baseline_pred": "SELECT s.FundingType FROM schools AS s JOIN satscores AS t ON s.CDSCode = t.cds WHERE s.City = 'Riverside' AND t.AvgScrMath > 400 GROUP BY s.FundingType HAVING AVG(t.AvgScrMath) > 400",
+      "alt_pred": "SELECT schools.FundingType FROM schools JOIN satscores ON schools.CDSCode = satscores.cds WHERE schools.City = 'Riverside' AND satscores.AvgScrMath > 400 GROUP BY schools.FundingType HAVING AVG(satscores.AvgScrMath) > 400",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 5526.180299999396
+    },
+    {
+      "question_id": 37,
+      "db_id": "california_schools",
+      "difficulty": "moderate",
+      "question": "What is the complete address of the school with the lowest excellence rate? Indicate the Street, City, Zip and State.",
+      "gold_sql": "SELECT T2.Street, T2.City, T2.State, T2.Zip FROM satscores AS T1 INNER JOIN schools AS T2 ON T1.cds = T2.CDSCode ORDER BY CAST(T1.NumGE1500 AS REAL) / T1.NumTstTakr ASC LIMIT 1",
+      "baseline_pred": "SELECT Street, City, Zip, State FROM schools WHERE CDSCode = (SELECT cds FROM satscores WHERE NumTstTakr > 0 ORDER BY (NumGE1500 * 1.0 / NumTstTakr) ASC LIMIT 1)",
+      "alt_pred": "SELECT schools.Street, schools.City, schools.Zip, schools.State FROM schools JOIN satscores ON schools.CDSCode = satscores.cds ORDER BY satscores.NumGE1500 * 1.0 / satscores.NumTstTakr ASC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2121.3300000017625
+    },
+    {
+      "question_id": 77,
+      "db_id": "california_schools",
+      "difficulty": "moderate",
+      "question": "Which schools served a grade span of Kindergarten to 9th grade in the county of Los Angeles and what is its Percent (%) Eligible FRPM (Ages 5-17)?",
+      "gold_sql": "SELECT T2.School, T1.`FRPM Count (Ages 5-17)` * 100 / T1.`Enrollment (Ages 5-17)` FROM frpm AS T1 INNER JOIN schools AS T2 ON T1.CDSCode = T2.CDSCode WHERE T2.County = 'Los Angeles' AND T2.GSserved = 'K-9'",
+      "baseline_pred": "SELECT f.`School Name`, (f.`FRPM Count (Ages 5-17)` / f.`Enrollment (Ages 5-17)`) * 100 AS `Percent (%) Eligible FRPM (Ages 5-17)` FROM frpm f JOIN schools s ON f.CDSCode = s.CDSCode WHERE f.`County Name` = 'Los Angeles' AND f.`Low Grade` = 'K' AND f.`High Grade` = '9'",
+      "alt_pred": "SELECT `School Name`, `Percent (%) Eligible FRPM (Ages 5-17)` FROM frpm WHERE `County Name` = 'Los Angeles' AND `Low Grade` = 'K' AND `High Grade` = '9'",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2690.3912000016135
+    },
+    {
+      "question_id": 125,
+      "db_id": "financial",
+      "difficulty": "challenging",
+      "question": "For loans contracts which are still running where client are in debt, list the district of the and the state the percentage unemployment rate increment from year 1995 to 1996.",
+      "gold_sql": "SELECT CAST((T3.A13 - T3.A12) AS REAL) * 100 / T3.A12 FROM loan AS T1 INNER JOIN account AS T2 ON T1.account_id = T2.account_id INNER JOIN district AS T3 ON T2.district_id = T3.district_id WHERE T1.status = 'D'",
+      "baseline_pred": "SELECT district.A2, CAST((district.A13 - district.A12) * 100.0 / district.A12 AS REAL) FROM loan JOIN account ON loan.account_id = account.account_id JOIN client ON account.district_id = client.district_id JOIN district ON client.district_id = district.district_id WHERE loan.status = 'D'",
+      "alt_pred": "SELECT district.A2, district.A3, ((district.A13 - district.A12) / district.A12) * 100 AS unemployment_increment FROM loan JOIN account ON loan.account_id = account.account_id JOIN disp ON account.account_id = disp.account_id AND disp.type = 'OWNER' JOIN client ON disp.client_id = client.client_id JOIN district ON account.district_id = district.district_id WHERE loan.status = 'D'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 34196.809699998994
+    },
+    {
+      "question_id": 173,
+      "db_id": "financial",
+      "difficulty": "challenging",
+      "question": "How often does account number 3 request an account statement to be released? What was the aim of debiting 3539 in total?",
+      "gold_sql": "SELECT T1.frequency, T2.k_symbol FROM account AS T1 INNER JOIN (SELECT account_id, k_symbol, SUM(amount) AS total_amount FROM `order` GROUP BY account_id, k_symbol) AS T2 ON T1.account_id = T2.account_id WHERE T1.account_id = 3 AND T2.total_amount = 3539",
+      "baseline_pred": "SELECT COUNT(*) AS frequency, k_symbol AS purpose FROM trans WHERE account_id = 3 AND type = 'VYDAJ' AND k_symbol = 'SLUZBY' GROUP BY k_symbol",
+      "alt_pred": "SELECT COUNT(*) FROM trans WHERE account_id = 3 AND k_symbol = 'SIPO'",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2783.9843000001565
+    },
+    {
+      "question_id": 207,
+      "db_id": "toxicology",
+      "difficulty": "challenging",
+      "question": "What elements are in a double type bond?",
+      "gold_sql": "SELECT DISTINCT T1.element FROM atom AS T1 INNER JOIN bond AS T2 ON T1.molecule_id = T2.molecule_id INNER JOIN connected AS T3 ON T1.atom_id = T3.atom_id WHERE T2.bond_type = '='",
+      "baseline_pred": "SELECT DISTINCT a.element FROM atom a JOIN bond b ON a.molecule_id = b.molecule_id JOIN connected c ON b.bond_id = c.bond_id WHERE b.bond_type = '='",
+      "alt_pred": "SELECT DISTINCT atom.element FROM atom JOIN connected ON atom.atom_id = connected.atom_id JOIN bond ON connected.bond_id = bond.bond_id WHERE bond.bond_type = '='",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1171.0190000012517
+    },
+    {
+      "question_id": 349,
+      "db_id": "card_games",
+      "difficulty": "moderate",
+      "question": "Name the card and artist with the most ruling information. Also state if the card is a promotional printing.",
+      "gold_sql": "SELECT T1.name, T1.artist, T1.isPromo FROM cards AS T1 INNER JOIN rulings AS T2 ON T1.uuid = T2.uuid WHERE T1.isPromo = 1 AND T1.artist = (SELECT artist FROM cards WHERE isPromo = 1 GROUP BY artist HAVING COUNT(DISTINCT uuid) = (SELECT MAX(count_uuid) FROM ( SELECT COUNT(DISTINCT uuid) AS count_uuid FROM cards WHERE isPromo = 1 GROUP BY artist ))) LIMIT 1",
+      "baseline_pred": "SELECT c.name, c.artist, c.isPromo FROM cards AS c JOIN rulings AS r ON c.uuid = r.uuid GROUP BY c.uuid ORDER BY COUNT(r.id) DESC LIMIT 1",
+      "alt_pred": "SELECT cards.name, cards.artist, cards.isPromo FROM cards JOIN rulings ON cards.uuid = rulings.uuid GROUP BY cards.uuid ORDER BY COUNT(rulings.uuid) DESC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 4558.712499998364
+    },
+    {
+      "question_id": 358,
+      "db_id": "card_games",
+      "difficulty": "simple",
+      "question": "What is the border color of card \"Ancestor's Chosen\"?",
+      "gold_sql": "SELECT DISTINCT borderColor FROM cards WHERE name = 'Ancestor''s Chosen'",
+      "baseline_pred": "SELECT borderColor FROM cards WHERE name = 'Ancestor''s Chosen'",
+      "alt_pred": "SELECT borderColor FROM cards WHERE name = 'Ancestor''s Chosen'",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1058.1645999991451
+    },
+    {
+      "question_id": 407,
+      "db_id": "card_games",
+      "difficulty": "moderate",
+      "question": "Lists all types of cards in German.",
+      "gold_sql": "SELECT T1.subtypes, T1.supertypes FROM cards AS T1 INNER JOIN foreign_data AS T2 ON T1.uuid = T2.uuid WHERE T2.language = 'German' AND T1.subtypes IS NOT NULL AND T1.supertypes IS NOT NULL",
+      "baseline_pred": "SELECT DISTINCT T1.subtypes, T1.supertypes FROM cards AS T1 INNER JOIN foreign_data AS T2 ON T1.uuid = T2.uuid WHERE T2.language = 'German' AND T1.subtypes IS NOT NULL AND T1.supertypes IS NOT NULL",
+      "alt_pred": "SELECT DISTINCT T1.subtypes, T1.supertypes FROM cards AS T1 INNER JOIN foreign_data AS T2 ON T1.uuid = T2.uuid WHERE T2.language = 'German' AND T1.subtypes IS NOT NULL AND T1.supertypes IS NOT NULL",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 4445.04379999853
+    },
+    {
+      "question_id": 408,
+      "db_id": "card_games",
+      "difficulty": "moderate",
+      "question": "How many unknown power cards contain info about the triggered ability",
+      "gold_sql": "SELECT Count(DISTINCT T1.id) FROM cards AS T1 INNER JOIN rulings AS T2 ON T1.uuid = T2.uuid WHERE (T1.power IS NULL OR T1.power = '*') AND T2.text LIKE '%triggered ability%'",
+      "baseline_pred": "SELECT COUNT(*) FROM cards WHERE (power IS NULL OR power = '*') AND text LIKE '%triggered ability%'",
+      "alt_pred": "SELECT COUNT(*) FROM cards WHERE (power IS NULL OR power = '*') AND text LIKE '%triggered ability%'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1400.4644000015105
+    },
+    {
+      "question_id": 484,
+      "db_id": "card_games",
+      "difficulty": "moderate",
+      "question": "Please list the Italian names of the cards in the set Coldsnap with the highest converted mana cost.",
+      "gold_sql": "SELECT T2.name FROM foreign_data AS T1 INNER JOIN cards AS T2 ON T2.uuid = T1.uuid INNER JOIN sets AS T3 ON T3.code = T2.setCode WHERE T3.name = 'Coldsnap' AND T1.language = 'Italian' ORDER BY T2.convertedManaCost DESC",
+      "baseline_pred": "SELECT fd.name FROM cards AS c JOIN foreign_data AS fd ON c.uuid = fd.uuid JOIN sets AS s ON c.setCode = s.code WHERE s.name = 'Coldsnap' AND fd.language = 'Italian' ORDER BY c.convertedManaCost DESC LIMIT 1",
+      "alt_pred": "SELECT foreign_data.name FROM cards JOIN foreign_data ON cards.uuid = foreign_data.uuid JOIN sets ON cards.setCode = sets.code WHERE sets.name = 'Coldsnap' AND foreign_data.language = 'Italian' ORDER BY cards.convertedManaCost DESC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 10665.936299999885
+    },
+    {
+      "question_id": 518,
+      "db_id": "card_games",
+      "difficulty": "moderate",
+      "question": "Which of the play format has the highest number of banned status? Indicate the play format and the names of all the card meet the condition.",
+      "gold_sql": "WITH MaxBanned AS (SELECT format, COUNT(*) AS count_banned FROM legalities WHERE status = 'Banned' GROUP BY format ORDER BY COUNT(*) DESC LIMIT 1) SELECT T2.format, T1.name FROM cards AS T1 INNER JOIN legalities AS T2 ON T2.uuid = T1.uuid INNER JOIN MaxBanned MB ON MB.format = T2.format WHERE T2.status = 'Banned'",
+      "baseline_pred": "SELECT l.format, c.name FROM legalities l JOIN cards c ON l.uuid = c.uuid WHERE l.status = 'Banned' GROUP BY l.format ORDER BY COUNT(l.status) DESC LIMIT 1",
+      "alt_pred": "SELECT cards.name FROM legalities JOIN cards ON legalities.uuid = cards.uuid WHERE legalities.status = 'Banned' AND legalities.format = (SELECT format FROM legalities WHERE status = 'Banned' GROUP BY format ORDER BY COUNT(*) DESC LIMIT 1)",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 4896.8562000009115
+    },
+    {
+      "question_id": 584,
+      "db_id": "codebase_community",
+      "difficulty": "moderate",
+      "question": "Write all the comments left by users who edited the post titled 'Why square the difference instead of taking the absolute value in standard deviation?'",
+      "gold_sql": "SELECT T2.Comment FROM posts AS T1 INNER JOIN postHistory AS T2 ON T1.Id = T2.PostId WHERE T1.Title = 'Why square the difference instead of taking the absolute value in standard deviation?'",
+      "baseline_pred": "SELECT comments.Text FROM comments JOIN posts ON comments.PostId = posts.Id WHERE posts.Title = 'Why square the difference instead of taking the absolute value in standard deviation?'",
+      "alt_pred": "SELECT comments.Text FROM comments JOIN posts ON comments.PostId = posts.Id WHERE posts.Title = 'Why square the difference instead of taking the absolute value in standard deviation?'",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 6172.021700000187
+    },
+    {
+      "question_id": 595,
+      "db_id": "codebase_community",
+      "difficulty": "moderate",
+      "question": "Which user have only one post history per post and having at least 1000 views?",
+      "gold_sql": "SELECT T2.UserId FROM users AS T1 INNER JOIN postHistory AS T2 ON T1.Id = T2.UserId INNER JOIN posts AS T3 ON T2.PostId = T3.Id WHERE T3.ViewCount >= 1000 GROUP BY T2.UserId HAVING COUNT(DISTINCT T2.PostHistoryTypeId) = 1",
+      "baseline_pred": "SELECT DISTINCT ph.UserId FROM postHistory ph JOIN posts p ON ph.PostId = p.Id WHERE p.ViewCount >= 1000 AND ph.UserId IS NOT NULL GROUP BY ph.UserId, ph.PostId HAVING COUNT(*) = 1",
+      "alt_pred": "SELECT users.Id FROM users JOIN posts ON users.Id = posts.OwnerUserId JOIN postHistory ON posts.Id = postHistory.PostId WHERE users.Views >= 1000 GROUP BY users.Id HAVING COUNT(postHistory.Id) = 1",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 7569.0351000012015
+    },
+    {
+      "question_id": 672,
+      "db_id": "codebase_community",
+      "difficulty": "moderate",
+      "question": "Among the users located in United Kingdom, how many users whose post have a total favorite amount of 4 or more?",
+      "gold_sql": "SELECT COUNT(T1.Id) FROM users AS T1 INNER JOIN posts AS T2 ON T1.Id = T2.OwnerUserId WHERE T1.Location = 'United Kingdom' AND T2.FavoriteCount >= 4",
+      "baseline_pred": "SELECT COUNT(DISTINCT u.Id) FROM users u JOIN posts p ON u.Id = p.OwnerUserId WHERE u.Location = 'United Kingdom' AND p.FavoriteCount >= 4",
+      "alt_pred": "SELECT COUNT(DISTINCT users.Id) FROM users INNER JOIN posts ON users.Id = posts.OwnerUserId WHERE users.Location = 'United Kingdom' AND posts.FavoriteCount >= 4",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1339.9495000012394
+    },
+    {
+      "question_id": 694,
+      "db_id": "codebase_community",
+      "difficulty": "moderate",
+      "question": "Provide the text of the latest 10 comments to the post with the title 'Analysing wind data with R' and the display name of the user who left it.",
+      "gold_sql": "SELECT T3.Text, T1.DisplayName FROM users AS T1 INNER JOIN posts AS T2 ON T1.Id = T2.OwnerUserId INNER JOIN comments AS T3 ON T2.Id = T3.PostId WHERE T2.Title = 'Analysing wind data with R' ORDER BY T1.CreationDate DESC LIMIT 10",
+      "baseline_pred": "SELECT comments.Text, comments.UserDisplayName FROM comments JOIN posts ON comments.PostId = posts.Id WHERE posts.Title = 'Analysing wind data with R' ORDER BY comments.CreationDate DESC LIMIT 10",
+      "alt_pred": "SELECT comments.Text, comments.UserDisplayName FROM comments JOIN posts ON comments.PostId = posts.Id WHERE posts.Title = 'Analysing wind data with R' ORDER BY comments.CreationDate DESC LIMIT 10",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1579.7966999998607
+    },
+    {
+      "question_id": 743,
+      "db_id": "superhero",
+      "difficulty": "challenging",
+      "question": "What is the percentage of superheroes who act in their own self-interest or make decisions based on their own moral code? Indicate how many of the said superheroes were published by Marvel Comics.",
+      "gold_sql": "SELECT (CAST(COUNT(*) AS REAL) * 100 / (SELECT COUNT(*) FROM superhero)), CAST(SUM(CASE WHEN T2.publisher_name = 'Marvel Comics' THEN 1 ELSE 0 END) AS REAL) FROM superhero AS T1 INNER JOIN publisher AS T2 ON T1.publisher_id = T2.id INNER JOIN alignment AS T3 ON T3.id = T1.alignment_id WHERE T3.alignment = 'Bad'",
+      "baseline_pred": "SELECT CAST(SUM(CASE WHEN a.alignment = 'Bad' THEN 1 ELSE 0 END) AS REAL) * 100 / COUNT(s.id) AS percentage, SUM(CASE WHEN a.alignment = 'Bad' AND p.publisher_name = 'Marvel Comics' THEN 1 ELSE 0 END) FROM superhero s INNER JOIN alignment a ON s.alignment_id = a.id LEFT JOIN publisher p ON s.publisher_id = p.id",
+      "alt_pred": "SELECT CAST(SUM(CASE WHEN alignment.alignment = 'Bad' THEN 1 ELSE 0 END) AS REAL) * 100 / COUNT(superhero.id) AS percentage, SUM(CASE WHEN alignment.alignment = 'Bad' AND publisher.publisher_name = 'Marvel Comics' THEN 1 ELSE 0 END) AS marvel_count FROM superhero JOIN alignment ON superhero.alignment_id = alignment.id JOIN publisher ON superhero.publisher_id = publisher.id",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2046.1073000005854
+    },
+    {
+      "question_id": 866,
+      "db_id": "formula_1",
+      "difficulty": "moderate",
+      "question": "Who was the player that got the lap time of 0:01:27 in the race No. 161? Show his introduction website.",
+      "gold_sql": "SELECT DISTINCT T2.forename, T2.surname, T2.url FROM lapTimes AS T1 INNER JOIN drivers AS T2 ON T2.driverId = T1.driverId WHERE T1.raceId = 161 AND T1.time LIKE '1:27%'",
+      "baseline_pred": "SELECT drivers.url FROM lapTimes JOIN drivers ON lapTimes.driverId = drivers.driverId WHERE lapTimes.raceId = 161 AND lapTimes.time LIKE '1:27%'",
+      "alt_pred": "SELECT drivers.url FROM drivers JOIN lapTimes ON drivers.driverId = lapTimes.driverId WHERE lapTimes.raceId = 161 AND lapTimes.time LIKE '1:27%'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1124.1408999994746
+    },
+    {
+      "question_id": 894,
+      "db_id": "formula_1",
+      "difficulty": "moderate",
+      "question": "What is the best lap time recorded? List the driver and race with such recorded lap time.",
+      "gold_sql": "SELECT T2.milliseconds, T1.forename, T1.surname, T3.name FROM drivers AS T1 INNER JOIN lapTimes AS T2 ON T1.driverId = T2.driverId INNER JOIN races AS T3 ON T2.raceId = T3.raceId ORDER BY T2.milliseconds ASC LIMIT 1",
+      "baseline_pred": "SELECT drivers.forename, drivers.surname, races.name FROM lapTimes JOIN drivers ON lapTimes.driverId = drivers.driverId JOIN races ON lapTimes.raceId = races.raceId WHERE lapTimes.milliseconds = (SELECT MIN(milliseconds) FROM lapTimes) LIMIT 1",
+      "alt_pred": "SELECT drivers.forename, drivers.surname, races.name FROM lapTimes JOIN drivers ON lapTimes.driverId = drivers.driverId JOIN races ON lapTimes.raceId = races.raceId WHERE lapTimes.milliseconds = (SELECT MIN(milliseconds) FROM lapTimes)",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2223.164500002895
+    },
+    {
+      "question_id": 896,
+      "db_id": "formula_1",
+      "difficulty": "challenging",
+      "question": "Calculate the percentage whereby Hamilton was not at the 1st track of the the f1 circuit since 2010.",
+      "gold_sql": "SELECT CAST(COUNT(CASE WHEN T2.position <> 1 THEN T2.position END) AS REAL) * 100 / COUNT(T2.driverStandingsId) FROM races AS T1 INNER JOIN driverStandings AS T2 ON T2.raceId = T1.raceId INNER JOIN drivers AS T3 ON T3.driverId = T2.driverId WHERE T3.surname = 'Hamilton' AND T1.year >= 2010",
+      "baseline_pred": "SELECT CAST(SUM(CASE WHEN r.positionOrder > 1 THEN 1 ELSE 0 END) AS REAL) * 100.0 / COUNT(*) FROM results r INNER JOIN drivers d ON r.driverId = d.driverId INNER JOIN races ra ON r.raceId = ra.raceId WHERE d.surname = 'Hamilton' AND ra.year >= 2010",
+      "alt_pred": "SELECT CAST(COUNT(CASE WHEN position > 1 THEN 1 END) AS REAL) * 100 / COUNT(*) FROM results JOIN drivers ON results.driverId = drivers.driverId JOIN races ON results.raceId = races.raceId WHERE drivers.surname = 'Hamilton' AND races.year >= 2010",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2146.8490999977803
+    },
+    {
+      "question_id": 902,
+      "db_id": "formula_1",
+      "difficulty": "simple",
+      "question": "Which race was Alex Yoong in when he was in track number less than 20?",
+      "gold_sql": "SELECT T1.name FROM races AS T1 INNER JOIN driverStandings AS T2 ON T2.raceId = T1.raceId INNER JOIN drivers AS T3 ON T3.driverId = T2.driverId WHERE T3.forename = 'Alex' AND T3.surname = 'Yoong' AND T2.position < 20",
+      "baseline_pred": "SELECT races.name FROM races JOIN results ON races.raceId = results.raceId JOIN drivers ON results.driverId = drivers.driverId WHERE drivers.forename = 'Alex' AND drivers.surname = 'Yoong' AND results.position < 20",
+      "alt_pred": "SELECT races.name FROM races JOIN results ON races.raceId = results.raceId JOIN drivers ON results.driverId = drivers.driverId WHERE drivers.forename = 'Alex' AND drivers.surname = 'Yoong' AND results.position < 20",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1287.2468999994453
+    },
+    {
+      "question_id": 930,
+      "db_id": "formula_1",
+      "difficulty": "simple",
+      "question": "In which Formula_1 race did Lewis Hamilton rank the highest?",
+      "gold_sql": "SELECT name FROM races WHERE raceId IN ( SELECT raceId FROM results WHERE rank = 1 AND driverId = ( SELECT driverId FROM drivers WHERE forename = 'Lewis' AND surname = 'Hamilton' ) )",
+      "baseline_pred": "SELECT races.name FROM races JOIN results ON races.raceId = results.raceId JOIN drivers ON results.driverId = drivers.driverId WHERE drivers.forename = 'Lewis' AND drivers.surname = 'Hamilton' ORDER BY results.rank ASC LIMIT 1",
+      "alt_pred": "SELECT races.name FROM races JOIN results ON races.raceId = results.raceId JOIN drivers ON results.driverId = drivers.driverId WHERE drivers.forename = 'Lewis' AND drivers.surname = 'Hamilton' ORDER BY results.rank ASC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1316.0355999971216
+    },
+    {
+      "question_id": 988,
+      "db_id": "formula_1",
+      "difficulty": "challenging",
+      "question": "List down top 3 German drivers who has the shortest average pit stop duration and were born between 1980-1985.",
+      "gold_sql": "SELECT T2.forename, T2.surname FROM pitStops AS T1 INNER JOIN drivers AS T2 on T1.driverId = T2.driverId WHERE T2.nationality = 'German' AND STRFTIME('%Y', T2.dob) BETWEEN '1980' AND '1985' GROUP BY T2.forename, T2.surname ORDER BY AVG(T1.duration) LIMIT 3",
+      "baseline_pred": "SELECT d.forename || ' ' || d.surname AS full_name FROM pitStops p JOIN drivers d ON p.driverId = d.driverId WHERE strftime('%Y', d.dob) BETWEEN '1980' AND '1985' AND d.nationality = 'German' GROUP BY d.driverId, d.forename, d.surname ORDER BY AVG(p.milliseconds) ASC LIMIT 3",
+      "alt_pred": "SELECT drivers.forename, drivers.surname FROM drivers JOIN pitStops ON drivers.driverId = pitStops.driverId WHERE drivers.nationality = 'German' AND strftime('%Y', drivers.dob) BETWEEN '1980' AND '1985' GROUP BY drivers.driverId ORDER BY AVG(pitStops.milliseconds) ASC LIMIT 3",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1455.3177999987383
+    },
+    {
+      "question_id": 990,
+      "db_id": "formula_1",
+      "difficulty": "challenging",
+      "question": "What is the constructor reference name of the champion in the 2009 Singapore Grand Prix? Please give its website.",
+      "gold_sql": "SELECT T3.constructorRef, T3.url FROM results AS T1 INNER JOIN races AS T2 on T1.raceId = T2.raceId INNER JOIN constructors AS T3 on T1.constructorId = T3.constructorId WHERE T2.name = 'Singapore Grand Prix' AND T2.year = 2009 AND T1.time LIKE '_:%:__.___'",
+      "baseline_pred": "SELECT constructors.constructorRef, constructors.url FROM results INNER JOIN races ON results.raceId = races.raceId INNER JOIN constructors ON results.constructorId = constructors.constructorId WHERE races.year = 2009 AND races.name = 'Singapore Grand Prix' AND results.positionOrder = 1 ORDER BY results.time DESC LIMIT 1",
+      "alt_pred": "SELECT constructors.constructorRef, constructors.url FROM constructors JOIN results ON constructors.constructorId = results.constructorId JOIN races ON results.raceId = races.raceId WHERE races.year = 2009 AND races.name = 'Singapore Grand Prix' AND results.position = 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2576.8675999970583
+    },
+    {
+      "question_id": 1029,
+      "db_id": "european_football_2",
+      "difficulty": "moderate",
+      "question": "What are the speed in which attacks are put together of the top 4 teams with the highest build Up Play Speed?",
+      "gold_sql": "SELECT t1.buildUpPlaySpeed FROM Team_Attributes AS t1 INNER JOIN Team AS t2 ON t1.team_api_id = t2.team_api_id ORDER BY t1.buildUpPlaySpeed ASC LIMIT 4",
+      "baseline_pred": "SELECT buildUpPlaySpeed FROM Team_Attributes ORDER BY buildUpPlaySpeed DESC LIMIT 4",
+      "alt_pred": "SELECT Team.team_long_name, Team_Attributes.buildUpPlaySpeed FROM Team JOIN Team_Attributes ON Team.team_api_id = Team_Attributes.team_api_id ORDER BY Team_Attributes.buildUpPlaySpeed DESC LIMIT 4",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2120.9820000003674
+    },
+    {
+      "question_id": 1036,
+      "db_id": "european_football_2",
+      "difficulty": "challenging",
+      "question": "List the long name of teams with above-average build-up play passing in 2012.",
+      "gold_sql": "SELECT DISTINCT t4.team_long_name FROM Team_Attributes AS t3 INNER JOIN Team AS t4 ON t3.team_api_id = t4.team_api_id WHERE SUBSTR(t3.`date`, 1, 4) = '2012' AND t3.buildUpPlayPassing > ( SELECT CAST(SUM(t2.buildUpPlayPassing) AS REAL) / COUNT(t1.id) FROM Team AS t1 INNER JOIN Team_Attributes AS t2 ON t1.team_api_id = t2.team_api_id WHERE STRFTIME('%Y',t2.`date`) = '2012')",
+      "baseline_pred": "SELECT T1.team_long_name FROM Team AS T1 INNER JOIN Team_Attributes AS T2 ON T1.team_api_id = T2.team_api_id WHERE strftime('%Y', T2.date) = '2012' AND T2.buildUpPlayPassing > (SELECT AVG(buildUpPlayPassing) FROM Team_Attributes WHERE buildUpPlayPassing IS NOT NULL AND strftime('%Y', date) = '2012')",
+      "alt_pred": "SELECT T1.team_long_name FROM Team AS T1 JOIN Team_Attributes AS T2 ON T1.team_api_id = T2.team_api_id WHERE T2.buildUpPlayPassing > (SELECT AVG(buildUpPlayPassing) FROM Team_Attributes WHERE buildUpPlayPassing IS NOT NULL AND strftime('%Y', date) = '2012') AND strftime('%Y', T2.date) = '2012'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1607.06680000294
+    },
+    {
+      "question_id": 1094,
+      "db_id": "european_football_2",
+      "difficulty": "challenging",
+      "question": "How much higher in percentage is Ariel Borysiuk's overall rating than that of Paulin Puel?",
+      "gold_sql": "SELECT (SUM(CASE WHEN t1.player_name = 'Ariel Borysiuk' THEN t2.overall_rating ELSE 0 END) * 1.0 - SUM(CASE WHEN t1.player_name = 'Paulin Puel' THEN t2.overall_rating ELSE 0 END)) * 100 / SUM(CASE WHEN t1.player_name = 'Paulin Puel' THEN t2.overall_rating ELSE 0 END) FROM Player AS t1 INNER JOIN Player_Attributes AS t2 ON t1.player_api_id = t2.player_api_id",
+      "baseline_pred": "SELECT CAST((MAX(CASE WHEN p.player_name = 'Ariel Borysiuk' THEN pa.overall_rating END) - MAX(CASE WHEN p.player_name = 'Paulin Puel' THEN pa.overall_rating END)) * 100.0 / MAX(CASE WHEN p.player_name = 'Paulin Puel' THEN pa.overall_rating END) AS REAL) AS percentage_higher FROM Player p INNER JOIN Player_Attributes pa ON p.player_api_id = pa.player_api_id WHERE p.player_name IN ('Ariel Borysiuk', 'Paulin Puel') AND pa.overall_rating IS NOT NULL",
+      "alt_pred": "SELECT (T1.overall_rating - T2.overall_rating) * 100.0 / T2.overall_rating AS percentage_higher FROM Player_Attributes AS T1 JOIN Player AS T2 ON T1.player_api_id = T2.player_api_id WHERE T2.player_name = 'Ariel Borysiuk' AND T1.player_name = 'Paulin Puel'",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 3268.8476000002993
+    },
+    {
+      "question_id": 1144,
+      "db_id": "european_football_2",
+      "difficulty": "simple",
+      "question": "Please state the finishing rate and curve score of the player who has the heaviest weight.",
+      "gold_sql": "SELECT id, finishing, curve FROM Player_Attributes WHERE player_api_id = ( SELECT player_api_id FROM Player ORDER BY weight DESC LIMIT 1 ) LIMIT 1",
+      "baseline_pred": "SELECT finishing, curve FROM Player_Attributes JOIN Player ON Player_Attributes.player_api_id = Player.player_api_id WHERE Player.weight = (SELECT MAX(weight) FROM Player)",
+      "alt_pred": "SELECT Player_Attributes.finishing, Player_Attributes.curve FROM Player_Attributes JOIN Player ON Player_Attributes.player_api_id = Player.player_api_id WHERE Player.weight = (SELECT MAX(weight) FROM Player)",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2071.239600001718
+    },
+    {
+      "question_id": 1168,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "challenging",
+      "question": "The oldest SJS patient's medical laboratory work was completed on what date, and what age was the patient when they initially arrived at the hospital?",
+      "gold_sql": "SELECT T1.Date, STRFTIME('%Y', T2.`First Date`) - STRFTIME('%Y', T2.Birthday),T2.Birthday FROM Laboratory AS T1 INNER JOIN Patient AS T2 ON T1.ID = T2.ID WHERE T2.Diagnosis = 'SJS' AND T2.Birthday IS NOT NULL ORDER BY T2.Birthday ASC LIMIT 1",
+      "baseline_pred": "SELECT L.Date, (strftime('%Y', P.\"First Date\") - strftime('%Y', P.Birthday)) FROM Patient P INNER JOIN Laboratory L ON P.ID = L.ID WHERE P.Diagnosis = 'SJS' AND P.Birthday = (SELECT MIN(Birthday) FROM Patient WHERE Diagnosis = 'SJS') ORDER BY L.Date ASC LIMIT 1",
+      "alt_pred": "SELECT Laboratory.Date, (STRFTIME('%Y', Patient.`First Date`) - STRFTIME('%Y', Patient.Birthday)) AS AgeAtAdmission FROM Examination JOIN Patient ON Examination.ID = Patient.ID JOIN Laboratory ON Examination.ID = Laboratory.ID WHERE Examination.Diagnosis = 'SjS' ORDER BY Patient.Birthday ASC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 3766.808599997603
+    },
+    {
+      "question_id": 1205,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "moderate",
+      "question": "Was the patient with the number 57266's uric acid within a normal range?",
+      "gold_sql": "SELECT CASE WHEN (T1.SEX = 'F' AND T2.UA > 6.5) OR (T1.SEX = 'M' AND T2.UA > 8.0) THEN true ELSE false END FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T1.ID = 57266",
+      "baseline_pred": "SELECT CASE WHEN (T1.UA > 8.0 AND T2.SEX = 'M') OR (T1.UA > 6.5 AND T2.SEX = 'F') THEN 'Yes' ELSE 'No' END AS is_normal FROM Laboratory AS T1 INNER JOIN Patient AS T2 ON T1.ID = T2.ID WHERE T1.ID = 57266 LIMIT 1",
+      "alt_pred": "SELECT CASE WHEN (T1.SEX = 'M' AND T2.UA > 8.0) OR (T1.SEX = 'F' AND T2.UA > 6.5) THEN 'Yes' ELSE 'No' END AS 'Uric Acid Normal' FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T1.ID = 57266",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 10207.424599997466
+    },
+    {
+      "question_id": 1235,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "moderate",
+      "question": "What are the patient's diagnosis for those who has lower red blood blood cell? State their ID and age.",
+      "gold_sql": "SELECT DISTINCT T1.Diagnosis, T1.ID , STRFTIME('%Y', CURRENT_TIMESTAMP) -STRFTIME('%Y', T1.Birthday) FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T2.RBC < 3.5",
+      "baseline_pred": "SELECT P.ID, (strftime('%Y', 'now') - strftime('%Y', P.Birthday)) AS age, P.Diagnosis FROM Patient AS P JOIN Laboratory AS L ON P.ID = L.ID WHERE L.RBC < 3.5",
+      "alt_pred": "SELECT Patient.ID, strftime('%Y', 'now') - strftime('%Y', Patient.Birthday) AS age, Patient.Diagnosis FROM Patient JOIN Laboratory ON Patient.ID = Laboratory.ID WHERE Laboratory.RBC < 3.5",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 3232.0370999987063
+    },
+    {
+      "question_id": 1247,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "challenging",
+      "question": "Among the male patients who have a normal level of white blood cells, how many of them have an abnormal fibrinogen level?",
+      "gold_sql": "SELECT COUNT(DISTINCT T1.ID) FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T2.FG <= 150 OR T2.FG >= 450 AND T2.WBC > 3.5 AND T2.WBC < 9.0 AND T1.SEX = 'M'",
+      "baseline_pred": "SELECT COUNT(DISTINCT L.ID) FROM Patient P INNER JOIN Laboratory L ON P.ID = L.ID WHERE P.SEX = 'M' AND L.WBC > 3.5 AND L.WBC < 9.0 AND (L.FG <= 150 OR L.FG >= 450)",
+      "alt_pred": "SELECT COUNT(DISTINCT T1.ID) FROM Examination AS T1 JOIN Patient AS T2 ON T1.ID = T2.ID JOIN Laboratory AS T3 ON T1.ID = T3.ID WHERE T2.SEX = 'M' AND T3.WBC > 3.5 AND T3.WBC < 9.0 AND (T3.FG <= 150 OR T3.FG >= 450)",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2186.199299998407
+    },
+    {
+      "question_id": 1251,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "simple",
+      "question": "How many patients with an Ig G higher than normal?",
+      "gold_sql": "SELECT COUNT(DISTINCT T1.ID) FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID INNER JOIN Examination AS T3 ON T3.ID = T2.ID WHERE T2.IGG >= 2000",
+      "baseline_pred": "SELECT COUNT(DISTINCT ID) FROM Laboratory WHERE IGG >= 2000",
+      "alt_pred": "SELECT COUNT(DISTINCT Patient.ID) FROM Patient JOIN Laboratory ON Patient.ID = Laboratory.ID WHERE Laboratory.IGG >= 2000",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2756.7782000005536
+    },
+    {
+      "question_id": 1254,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "moderate",
+      "question": "How many patients with a normal Ig A level came to the hospital after 1990/1/1?",
+      "gold_sql": "SELECT COUNT(T1.ID) FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T2.IGA BETWEEN 80 AND 500 AND  strftime('%Y',  T1.`First Date`) > '1990'",
+      "baseline_pred": "SELECT COUNT(DISTINCT P.ID) FROM Patient P JOIN Laboratory L ON P.ID = L.ID WHERE L.IGA > 80 AND L.IGA < 500 AND P.`First Date` >= '1990-01-01'",
+      "alt_pred": "SELECT COUNT(DISTINCT Patient.ID) FROM Patient JOIN Examination ON Patient.ID = Examination.ID WHERE Examination.`aCL IgA` > 80 AND Examination.`aCL IgA` < 500 AND Patient.`First Date` >= '1990-01-01'",
+      "alt_confidence": 1.0,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 3027.875600000698
+    },
+    {
+      "question_id": 1275,
+      "db_id": "thrombosis_prediction",
+      "difficulty": "moderate",
+      "question": "Among the patients who has a normal level of anti-centromere and a normal level of anti-SSB, how many of them are male?",
+      "gold_sql": "SELECT COUNT(DISTINCT T1.ID) FROM Patient AS T1 INNER JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T2.CENTROMEA IN ('negative', '0') AND T2.SSB IN ('negative', '0') AND T1.SEX = 'M'",
+      "baseline_pred": "SELECT COUNT(DISTINCT P.ID) FROM Patient P JOIN Examination E ON P.ID = E.ID WHERE E.CENTROMEA IN ('-', '+-') AND E.SSB IN ('-', '+-') AND P.SEX = 'M'",
+      "alt_pred": "SELECT COUNT(DISTINCT T1.ID) FROM Patient AS T1 JOIN Laboratory AS T2 ON T1.ID = T2.ID WHERE T2.CENTROMEA IN ('-', '+-') AND T2.SSB IN ('-', '+-') AND T1.SEX = 'M'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1953.2646000006935
+    },
+    {
+      "question_id": 1399,
+      "db_id": "student_club",
+      "difficulty": "moderate",
+      "question": "Did Maya Mclean attend the 'Women's Soccer' event?",
+      "gold_sql": "SELECT CASE WHEN T3.event_name = 'Women''s Soccer' THEN 'YES' END AS result FROM member AS T1 INNER JOIN attendance AS T2 ON T1.member_id = T2.link_to_member INNER JOIN event AS T3 ON T2.link_to_event = T3.event_id WHERE T1.first_name = 'Maya' AND T1.last_name = 'Mclean'",
+      "baseline_pred": "SELECT COUNT(*) > 0 AS attended FROM attendance AS a JOIN member AS m ON a.link_to_member = m.member_id JOIN event AS e ON a.link_to_event = e.event_id WHERE m.first_name = 'Maya' AND m.last_name = 'Mclean' AND e.event_name = 'Women''s Soccer'",
+      "alt_pred": "SELECT COUNT(*) FROM attendance JOIN member ON attendance.link_to_member = member.member_id JOIN event ON attendance.link_to_event = event.event_id WHERE member.first_name = 'Maya' AND member.last_name = 'Mclean' AND event.event_name = 'Women''s Soccer'",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2330.6733000026725
+    },
+    {
+      "question_id": 1404,
+      "db_id": "student_club",
+      "difficulty": "moderate",
+      "question": "Identify the type of expenses and their total value approved for 'October Meeting' event.",
+      "gold_sql": "SELECT T1.type, SUM(T3.cost) FROM event AS T1 INNER JOIN budget AS T2 ON T1.event_id = T2.link_to_event INNER JOIN expense AS T3 ON T2.budget_id = T3.link_to_budget WHERE T1.event_name = 'October Meeting'",
+      "baseline_pred": "SELECT expense.expense_description AS type, SUM(expense.cost) AS total_value FROM expense JOIN budget ON expense.link_to_budget = budget.budget_id JOIN event ON budget.link_to_event = event.event_id WHERE event.event_name = 'October Meeting' AND expense.approved = 'true' GROUP BY expense.expense_description",
+      "alt_pred": "SELECT expense.expense_description, SUM(expense.cost) AS total_value FROM expense JOIN budget ON expense.link_to_budget = budget.budget_id JOIN event ON budget.link_to_event = event.event_id WHERE event.event_name = 'October Meeting' AND expense.approved = 'true' GROUP BY expense.expense_description",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 1393.583299999591
+    },
+    {
+      "question_id": 1525,
+      "db_id": "debit_card_specializing",
+      "difficulty": "simple",
+      "question": "What is the percentage of the customers who used EUR in 2012/8/25?",
+      "gold_sql": "SELECT CAST(SUM(IIF(T2.Currency = 'EUR', 1, 0)) AS FLOAT) * 100 / COUNT(T1.CustomerID) FROM transactions_1k AS T1 INNER JOIN customers AS T2 ON T1.CustomerID = T2.CustomerID WHERE T1.Date = '2012-08-25'",
+      "baseline_pred": "SELECT CAST(COUNT(DISTINCT t.CustomerID) AS REAL) * 100 / (SELECT COUNT(*) FROM customers) FROM transactions_1k t JOIN customers c ON t.CustomerID = c.CustomerID WHERE t.Date = '2012-08-25' AND c.Currency = 'EUR'",
+      "alt_pred": "SELECT CAST(SUM(IIF(customers.Currency = 'EUR', 1, 0)) AS REAL) * 100 / COUNT(customers.CustomerID) FROM customers JOIN transactions_1k ON customers.CustomerID = transactions_1k.CustomerID WHERE transactions_1k.Date = '2012-08-25'",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": true,
+      "vote_match": true,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2390.831500000786
+    },
+    {
+      "question_id": 1529,
+      "db_id": "debit_card_specializing",
+      "difficulty": "moderate",
+      "question": "What is the amount spent by customer \"38508\" at the gas stations? How much had the customer spent in January 2012?",
+      "gold_sql": "SELECT SUM(T1.Price ) , SUM(IIF(T3.Date = '201201', T1.Price, 0)) FROM transactions_1k AS T1 INNER JOIN gasstations AS T2 ON T1.GasStationID = T2.GasStationID INNER JOIN yearmonth AS T3 ON T1.CustomerID = T3.CustomerID WHERE T1.CustomerID = '38508'",
+      "baseline_pred": "SELECT SUM(Amount) AS TotalSpent FROM transactions_1k WHERE CustomerID = 38508 AND GasStationID IS NOT NULL UNION ALL SELECT SUM(Consumption) AS TotalSpent FROM yearmonth WHERE CustomerID = 38508 AND Date = '201201'",
+      "alt_pred": "SELECT SUM(transactions_1k.Amount) AS total_amount_spent FROM transactions_1k WHERE transactions_1k.CustomerID = 38508 AND transactions_1k.Date LIKE '2012-01%'",
+      "alt_confidence": 0.8,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2107.02289999972
+    },
+    {
+      "question_id": 1531,
+      "db_id": "debit_card_specializing",
+      "difficulty": "moderate",
+      "question": "Who is the top spending customer and how much is the average price per single item purchased by this customer? What currency was being used?",
+      "gold_sql": "SELECT T2.CustomerID, SUM(T2.Price / T2.Amount), T1.Currency FROM customers AS T1 INNER JOIN transactions_1k AS T2 ON T1.CustomerID = T2.CustomerID WHERE T2.CustomerID = ( SELECT CustomerID FROM yearmonth ORDER BY Consumption DESC LIMIT 1 ) GROUP BY T2.CustomerID, T1.Currency",
+      "baseline_pred": "SELECT customers.CustomerID, SUM(transactions_1k.Price) / SUM(transactions_1k.Amount) AS avg_price_per_item, customers.Currency FROM transactions_1k JOIN customers ON transactions_1k.CustomerID = customers.CustomerID GROUP BY transactions_1k.CustomerID, customers.Currency ORDER BY SUM(transactions_1k.Price) DESC LIMIT 1",
+      "alt_pred": "SELECT customers.CustomerID, customers.Currency, SUM(transactions_1k.Price) / SUM(transactions_1k.Amount) AS avg_price_per_item FROM transactions_1k JOIN customers ON transactions_1k.CustomerID = customers.CustomerID GROUP BY transactions_1k.CustomerID, customers.Currency ORDER BY SUM(transactions_1k.Price) DESC LIMIT 1",
+      "alt_confidence": 0.9,
+      "baseline_match": false,
+      "alt_match": false,
+      "vote_match": false,
+      "vote_source": "critique-retry",
+      "elapsed_ms": 2250.0837999978103
+    }
+  ]
+}

src/nl_sql/agent/nodes/_support.py

@@ -56,6 +56,61 @@ def parse_generate_sql_output(text: str) -> GenerateSQLOutput:
     )
 
 
+_M_COL_RE = re.compile(
+    r"  - (?P<col>[^:]+?):\s+(?P<type>[A-Za-z][A-Za-z0-9_()]*)\s+\[(?P<flags>[^\]]*)\]"
+    r"(?:\s*\|\s*nulls=\d+(?:\s*\([^)]+\))?,\s*distinct=\d+)?"
+    r"(?:\s*\|\s*samples:\s*(?P<samples>.+))?$"
+)
+_M_FK_RE = re.compile(r"  - \(([^)]+)\) -> (\S+?)\(([^)]+)\)")
+
+
+def render_m_schema(context: ContextBundle | None) -> str:
+    """Compact M-Schema rendering (XiYan-SQL style) parsed from chunk text.
+
+    Replaces verbose table-card dump with: ``table.column (type) [samples]``
+    per line plus a trailing FK block. Reduces tokens by ~60% and surfaces
+    FK pairs as first-class signal next to columns instead of buried inside
+    multi-section cards.
+    """
+    if context is None:
+        return "(no schema context)"
+    all_hits = list(context.schema_hits) + list(context.fk_neighbours)
+    all_hits.sort(key=lambda h: h.table_name.lower())
+    if not all_hits:
+        return "(no tables matched)"
+    col_lines: list[str] = []
+    fk_lines: list[str] = []
+    for hit in all_hits:
+        table = hit.table_name
+        for raw_line in hit.text.splitlines():
+            m = _M_COL_RE.match(raw_line)
+            if m:
+                col = m.group("col").strip()
+                col_type = m.group("type")
+                flags = (m.group("flags") or "").strip()
+                samples = (m.group("samples") or "").strip()
+                pk = "PK" in flags.split()
+                parts = [f"{table}.{col} ({col_type})"]
+                if pk:
+                    parts.append("[PK]")
+                if samples:
+                    parts.append(f"[{samples}]")
+                col_lines.append(" ".join(parts))
+                continue
+            fk_m = _M_FK_RE.match(raw_line)
+            if fk_m:
+                local_cols, ref_table, ref_cols = fk_m.groups()
+                fk_lines.append(f"{table}.({local_cols}) -> {ref_table}.({ref_cols})")
+    blocks: list[str] = ["# Columns", *col_lines] if col_lines else ["(no columns parsed)"]
+    if fk_lines:
+        blocks.append("\n# Foreign keys")
+        blocks.extend(fk_lines)
+    appendix = _render_extended_samples_appendix(context.extended_samples)
+    if appendix:
+        blocks.append(appendix)
+    return "\n".join(blocks)
+
+
 def render_schema_block(
     context: ContextBundle | None,
     *,

src/nl_sql/agent/nodes/generate_sql.py

@@ -1,67 +1,76 @@
-"""Node: ask codestral (or any LLMProvider) for SQL given the schema context.
-
-Builds the prompt from the active context bundle, dispatches to the provider,
-parses the JSON response into a `GenerateSQLOutput`. The same node powers the
-*initial* generation pass — the repair pass is a separate node that calls
-this same provider with a different prompt.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable
-
-from nl_sql.agent.nodes._support import (
-    parse_generate_sql_output,
-    render_fewshot_block,
-    render_schema_block,
-)
-from nl_sql.agent.prompts import load_prompt
-from nl_sql.agent.state import PipelineState
-from nl_sql.llm.providers.base import GenerateRequest, LLMProvider
-
-
-def make_generate_sql_node(
-    provider: LLMProvider,
-    *,
-    max_tokens: int = 1024,
-    temperature: float = 0.0,
-    sort_schema_block: bool = False,
-) -> Callable[[PipelineState], PipelineState]:
-    def node(state: PipelineState) -> PipelineState:
-        question = state.get("question", "")
-        dialect = state.get("dialect", "sqlite")
-        context = state.get("context")
-        plan_raw = (state.get("plan") or "").strip()
-        plan_block = plan_raw if plan_raw else "(no plan — generate SQL directly from question)"
-        prompt = load_prompt(
-            "generate_sql",
-            dialect=dialect,
-            schema_block=render_schema_block(context, sort_alphabetically=sort_schema_block),
-            fewshot_block=render_fewshot_block(context),
-            plan_block=plan_block,
-            question=question,
-        )
-        response = provider.generate(
-            GenerateRequest(prompt=prompt, max_tokens=max_tokens, temperature=temperature)
-        )
-        parsed = parse_generate_sql_output(response.text)
-        trace = list(state.get("trace") or [])
-        trace.append(
-            {
-                "node": "generate_sql",
-                "model": response.model,
-                "confidence": parsed.confidence,
-                "tables_used": list(parsed.tables_used),
-                "input_tokens": response.input_tokens,
-                "output_tokens": response.output_tokens,
-            }
-        )
-        # Reset any stale outcome / error from a previous repair iteration.
-        return {
-            "generated": parsed,
-            "outcome": None,
-            "last_error": "",
-            "trace": trace,
-        }
-
-    return node
+"""Node: ask codestral (or any LLMProvider) for SQL given the schema context.
+
+Builds the prompt from the active context bundle, dispatches to the provider,
+parses the JSON response into a `GenerateSQLOutput`. The same node powers the
+*initial* generation pass — the repair pass is a separate node that calls
+this same provider with a different prompt.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+
+from nl_sql.agent.nodes._support import (
+    parse_generate_sql_output,
+    render_fewshot_block,
+    render_m_schema,
+    render_schema_block,
+)
+from nl_sql.agent.prompts import load_prompt
+from nl_sql.agent.state import PipelineState
+from nl_sql.llm.providers.base import GenerateRequest, LLMProvider
+
+
+def make_generate_sql_node(
+    provider: LLMProvider,
+    *,
+    max_tokens: int = 1024,
+    temperature: float = 0.0,
+    sort_schema_block: bool = False,
+) -> Callable[[PipelineState], PipelineState]:
+    def node(state: PipelineState) -> PipelineState:
+        question = state.get("question", "")
+        dialect = state.get("dialect", "sqlite")
+        context = state.get("context")
+        plan_raw = (state.get("plan") or "").strip()
+        plan_block = plan_raw if plan_raw else "(no plan — generate SQL directly from question)"
+        # Experimental: M-Schema serialization (XiYan-SQL style) — compact
+        # one-line-per-column with inline samples + trailing FK pairs block.
+        # Toggle via env NLSQL_M_SCHEMA=1 to A/B against verbose card layout.
+        import os
+        if os.environ.get("NLSQL_M_SCHEMA") == "1":
+            schema_text = render_m_schema(context)
+        else:
+            schema_text = render_schema_block(context, sort_alphabetically=sort_schema_block)
+        prompt = load_prompt(
+            "generate_sql",
+            dialect=dialect,
+            schema_block=schema_text,
+            fewshot_block=render_fewshot_block(context),
+            plan_block=plan_block,
+            question=question,
+        )
+        response = provider.generate(
+            GenerateRequest(prompt=prompt, max_tokens=max_tokens, temperature=temperature)
+        )
+        parsed = parse_generate_sql_output(response.text)
+        trace = list(state.get("trace") or [])
+        trace.append(
+            {
+                "node": "generate_sql",
+                "model": response.model,
+                "confidence": parsed.confidence,
+                "tables_used": list(parsed.tables_used),
+                "input_tokens": response.input_tokens,
+                "output_tokens": response.output_tokens,
+            }
+        )
+        # Reset any stale outcome / error from a previous repair iteration.
+        return {
+            "generated": parsed,
+            "outcome": None,
+            "last_error": "",
+            "trace": trace,
+        }
+
+    return node