| Based on svenbl80/deberta-v3-Base-finetuned-mnli finetuned on a synthetic dataset (labels) | |
| Performance on test dataset: | |
| precision recall f1-score support | |
| 0 0.99 1.00 0.99 94 | |
| 1 1.00 1.00 1.00 28 | |
| 2 1.00 0.98 0.99 66 | |
| accuracy 0.99 188 | |
| macro avg 1.00 0.99 1.00 188 | |
| weighted avg 0.99 0.99 0.99 188 | |
| Performance on real estate benchmark: | |
| precision recall f1-score support | |
| 0 0.30 0.45 0.36 100 | |
| 1 0.21 0.15 0.18 100 | |
| 2 0.35 0.27 0.31 100 | |
| accuracy 0.29 300 | |
| macro avg 0.29 0.29 0.28 300 | |
| weighted avg 0.29 0.29 0.28 300 | |
| Baseline (svenbl80/deberta-v3-Base-finetuned-mnli) for real estate benchmark: | |
| 0 0.89 0.68 0.77 100 | |
| 1 0.63 0.92 0.75 100 | |
| 2 0.88 0.69 0.78 100 | |
| accuracy 0.76 300 | |
| macro avg 0.80 0.76 0.77 300 | |
| weighted avg 0.80 0.76 0.77 300 |