onkolahmet commited on
Commit
c21a390
·
verified ·
1 Parent(s): abc576f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -3
README.md CHANGED
@@ -80,15 +80,27 @@ The model was evaluated using standard text generation metrics (BLEU, ROUGE-L F1
80
  ### 🔹 Few-shot Prompting (on `gretelai/synthetic_text_to_sql/test`)
81
 
82
  **After Post-processing:**
83
- - **BLEU Score:** 0.2893
84
- - **ROUGE-L F1:** 0.5233
85
- - **CHRF Score:** 58.9214
86
 
87
  **Before Post-processing:**
88
  - **BLEU Score:** 0.1272
89
  - **ROUGE-L F1:** 0.2816
90
  - **CHRF Score:** 46.1643
91
 
 
 
 
 
 
 
 
 
 
 
 
 
92
  ## Citations
93
 
94
  Cite TRL as:
 
80
  ### 🔹 Few-shot Prompting (on `gretelai/synthetic_text_to_sql/test`)
81
 
82
  **After Post-processing:**
83
+ - **BLEU Score:** 0.2680
84
+ - **ROUGE-L F1:** 0.4975
85
+ - **CHRF Score:** 57.1704
86
 
87
  **Before Post-processing:**
88
  - **BLEU Score:** 0.1272
89
  - **ROUGE-L F1:** 0.2816
90
  - **CHRF Score:** 46.1643
91
 
92
+
93
+ **SQL-Specific Metrics:**
94
+ - **Exact Match (case insensitive):** 0.0000
95
+ - **Normalized Exact Match:** 0.0000
96
+ - **Average Component Match:** 0.2140
97
+ - **Average Entity Match:** 0.8067
98
+
99
+ **Query Quality Distribution:**
100
+ - **High Quality (≥80% component match):** 4 (4.0%)
101
+ - **Medium Quality (50-79% component match):** 17 (17.0%)
102
+ - **Low Quality (<50% component match):** 79 (79.0%)
103
+
104
  ## Citations
105
 
106
  Cite TRL as: