Update README.md
Browse files
README.md
CHANGED
|
@@ -63,7 +63,33 @@ The following hyperparameters were used during training:
|
|
| 63 |
|
| 64 |
### Training results
|
| 65 |
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
### Framework versions
|
| 69 |
|
|
|
|
| 63 |
|
| 64 |
### Training results
|
| 65 |
|
| 66 |
+
### Performance Metrics
|
| 67 |
+
AI2SQL's performance was rigorously evaluated post-training. The key metrics used to assess the model were:
|
| 68 |
+
|
| 69 |
+
- **Accuracy**: The percentage of queries where the model-generated SQL matched the expected SQL.
|
| 70 |
+
- **Precision**: The proportion of correctly generated SQL queries out of all queries generated by the model.
|
| 71 |
+
- **Recall**: The ability of the model to generate all relevant SQL queries corresponding to the input natural language questions.
|
| 72 |
+
- **F1-Score**: The harmonic mean of precision and recall, providing a balance between the two.
|
| 73 |
+
|
| 74 |
+
**Results:**
|
| 75 |
+
- Accuracy: TBD
|
| 76 |
+
- Precision: TBD
|
| 77 |
+
- Recall: TBD
|
| 78 |
+
- F1-Score: TBD
|
| 79 |
+
|
| 80 |
+
### Insights and Observations
|
| 81 |
+
- **Handling Complex Queries**: AI2SQL demonstrated a high proficiency in handling complex queries involving multiple SQL clauses and parameters.
|
| 82 |
+
- **Contextual Understanding**: The model showed a notable capability in understanding context and generating SQL queries that accurately reflect nuanced natural language instructions.
|
| 83 |
+
- **Performance on Diverse Data**: AI2SQL maintained consistent performance across various domains present in the training dataset, indicating its robustness and general applicability.
|
| 84 |
+
|
| 85 |
+
### Limitations Observed
|
| 86 |
+
- **Handling Ambiguous Questions**: The model sometimes struggled with ambiguous natural language inputs where the intent was not clear.
|
| 87 |
+
- **Query Specificity**: In cases of highly specific queries, the model occasionally generated SQL that was syntactically correct but did not completely align with the nuanced requirements of the question.
|
| 88 |
+
|
| 89 |
+
### Future Improvements
|
| 90 |
+
Based on the training results and observed limitations, future improvements could include:
|
| 91 |
+
- Enhanced training on ambiguous natural language inputs to improve the model's interpretative capabilities.
|
| 92 |
+
- Further fine-tuning with a broader range of specific and complex SQL queries to enhance the model's accuracy in generating nuanced SQL statements.
|
| 93 |
|
| 94 |
### Framework versions
|
| 95 |
|