Upload .\docs\evaluation.md with huggingface_hub
Browse files- .//docs//evaluation.md +18 -0
.//docs//evaluation.md
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Evaluation
|
| 2 |
+
|
| 3 |
+
## Metrics
|
| 4 |
+
|
| 5 |
+
| Metric | Purpose | Library |
|
| 6 |
+
|--------|---------|---------|
|
| 7 |
+
| Perplexity | LM quality | torch |
|
| 8 |
+
| ROUGE | Summarization | rouge-score |
|
| 9 |
+
| BLEU | Translation | nltk |
|
| 10 |
+
| Accuracy | Exact match | sklearn |
|
| 11 |
+
|
| 12 |
+
## Running
|
| 13 |
+
|
| 14 |
+
Evaluation runs automatically after training, or standalone:
|
| 15 |
+
|
| 16 |
+
```bash
|
| 17 |
+
python -c "from src.evaluation.metrics import BwengeEvaluator; ..."
|
| 18 |
+
```
|