Add model card with evaluation matrix, confusion matrix, and within-1 metrics
Browse files
README.md
CHANGED
|
@@ -11,20 +11,26 @@ This is a fine-tuned version of `unsloth/llama-3-8b-instruct-bnb-4bit` for CEFR-
|
|
| 11 |
- Training Args: learning_rate=2e-5, batch_size=8, epochs=0.1, cosine scheduler
|
| 12 |
- Optimizer: adamw_8bit
|
| 13 |
- Early Stopping: Patience=3, threshold=0.01
|
| 14 |
-
- **Evaluation Metrics**:
|
| 15 |
- CEFR Classifier Accuracy: 0.250
|
| 16 |
- Precision (Macro): 0.130
|
| 17 |
- Recall (Macro): 0.250
|
| 18 |
- F1-Score (Macro): 0.153
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
- Perplexity: 14.218
|
| 20 |
- Diversity (Unique Sentences): 0.933
|
| 21 |
-
- Inference Time (ms):
|
| 22 |
- Model Size (GB): 4.8
|
| 23 |
- Robustness (F1): 0.145
|
| 24 |
-
- **Confusion Matrix**:
|
| 25 |
- CSV: [confusion_matrix.csv](confusion_matrix.csv)
|
| 26 |
- Image: [confusion_matrix.png](confusion_matrix.png)
|
| 27 |
-
- **Per-Class Confusion Metrics**:
|
| 28 |
- A1: TP=0, FP=2, FN=10, TN=48
|
| 29 |
- A2: TP=0, FP=0, FN=10, TN=50
|
| 30 |
- B1: TP=10, FP=29, FN=0, TN=21
|
|
|
|
| 11 |
- Training Args: learning_rate=2e-5, batch_size=8, epochs=0.1, cosine scheduler
|
| 12 |
- Optimizer: adamw_8bit
|
| 13 |
- Early Stopping: Patience=3, threshold=0.01
|
| 14 |
+
- **Evaluation Metrics (Exact Matches)**:
|
| 15 |
- CEFR Classifier Accuracy: 0.250
|
| 16 |
- Precision (Macro): 0.130
|
| 17 |
- Recall (Macro): 0.250
|
| 18 |
- F1-Score (Macro): 0.153
|
| 19 |
+
- **Evaluation Metrics (Within ±1 Level)**:
|
| 20 |
+
- CEFR Classifier Accuracy: 0.733
|
| 21 |
+
- Precision (Macro): 0.701
|
| 22 |
+
- Recall (Macro): 0.733
|
| 23 |
+
- F1-Score (Macro): 0.687
|
| 24 |
+
- **Other Metrics**:
|
| 25 |
- Perplexity: 14.218
|
| 26 |
- Diversity (Unique Sentences): 0.933
|
| 27 |
+
- Inference Time (ms): 2208.386
|
| 28 |
- Model Size (GB): 4.8
|
| 29 |
- Robustness (F1): 0.145
|
| 30 |
+
- **Confusion Matrix (Exact Matches)**:
|
| 31 |
- CSV: [confusion_matrix.csv](confusion_matrix.csv)
|
| 32 |
- Image: [confusion_matrix.png](confusion_matrix.png)
|
| 33 |
+
- **Per-Class Confusion Metrics (Exact Matches)**:
|
| 34 |
- A1: TP=0, FP=2, FN=10, TN=48
|
| 35 |
- A2: TP=0, FP=0, FN=10, TN=50
|
| 36 |
- B1: TP=10, FP=29, FN=0, TN=21
|