rafmacalaba commited on
Commit
cd94272
·
verified ·
1 Parent(s): d2803e8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +31 -0
README.md CHANGED
@@ -63,3 +63,34 @@ F0.5 by threshold (sweet spots side-by-side):
63
  | 0.6 | 0.8506 | 0.8523 |
64
  | 0.7 | 0.8422 | 0.8634 |
65
  <!-- NER_COMPARISON_END -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  | 0.6 | 0.8506 | 0.8523 |
64
  | 0.7 | 0.8422 | 0.8634 |
65
  <!-- NER_COMPARISON_END -->
66
+
67
+
68
+ ## FCV cross-corpus validation (precision)
69
+
70
+ Validated on the World Bank Fragility, Conflict & Violence (FCV) document corpus with
71
+ the `gpt-5.6-luna` judge (Luna validation). Precision-only: there is no gold set of all
72
+ true mentions, so recall / F1 are not reported here.
73
+
74
+ ### Precision by config × label
75
+
76
+ | config | NAMED_DATA | DESCRIPTIVE_DATA | VAGUE_DATA | overall |
77
+ | --- | --- | --- | --- | --- |
78
+ | fcv_pads_east_asia | 80.1% | 81.6% | 91.8% | 81.9% |
79
+ | jdc_operational | 86.8% | 91.3% | 91.5% | 89.1% |
80
+ | refugee_pads | 83.8% | 92.8% | 91.9% | 89.1% |
81
+ | reliefweb | 83.7% | 85.4% | 95.5% | 85.6% |
82
+
83
+ ### Specificity agreement (extractor label == judge specificity)
84
+
85
+ | config | NAMED_DATA | DESCRIPTIVE_DATA | VAGUE_DATA |
86
+ | --- | --- | --- | --- |
87
+ | fcv_pads_east_asia | 82.8% | 88.3% | 82.1% |
88
+ | jdc_operational | 90.2% | 83.2% | 71.7% |
89
+ | refugee_pads | 89.4% | 86.5% | 77.5% |
90
+ | reliefweb | 92.6% | 82.5% | 73.4% |
91
+
92
+ Notes:
93
+ - `VAGUE_DATA` is the safest label (91–95% precision); `NAMED_DATA` is the weakest (80–87%).
94
+ - The extractor under-specifies: `VAGUE_DATA` is often judged `descriptive`, and
95
+ `DESCRIPTIVE_DATA` → `named` leaks occur.
96
+ - `reliefweb` carries more bad-JSON judge errors (OCR/HTML artifacts) than the other configs.