lfm2.5-350M-datause-provenance
LoRA SFT of LiquidAI/LFM2.5-350M for data-mention provenance attribute extraction
(producer / year / geography / acronym, verbatim from context).
Training
- base model:
LiquidAI/LFM2.5-350M - dataset:
rafmacalaba/data-use-provenance-sft - epochs: 3
- learning rate: 0.0002
- LoRA: r=16 alpha=32 dropout=0.05
- completion-only masking (loss on assistant JSON turn)
Evaluation (holdout, n=3487)
Exact string match of each emitted attribute against the gold label.
| attribute | tp | fp | fn | precision | recall | f0.5 | f1 |
|---|---|---|---|---|---|---|---|
| producer | 1143 | 296 | 306 | 0.7943 | 0.7888 | 0.7932 | 0.7916 |
| year | 1085 | 207 | 265 | 0.8398 | 0.8037 | 0.8323 | 0.8213 |
| geography | 1715 | 371 | 312 | 0.8221 | 0.8461 | 0.8268 | 0.8339 |
| acronym | 1271 | 212 | 145 | 0.8570 | 0.8976 | 0.8649 | 0.8769 |
| overall | 5214 | 1086 | 1028 | 0.8276 | 0.8353 | 0.8291 | 0.8314 |
Verbatim rate (emitted values that are substrings of the context): 6290/6300 = 0.9984
Sample predictions (holdout)
| mention | target | predicted | correct | verbatim |
|---|---|---|---|---|
| plant-level data | {"geography": "Indonesian"} | {"geography": "Indonesian"} | geography | geography |
| GIDD database | {"geography": "all regions in the world"} | {"acronym": "GIDD", "geography": "all regions in the world"} | geography | geography,acronym |
| NFHS-3 data | {"acronym": "NFHS-3", "geography": "India"} | {"acronym": "NFHS-3", "geography": "India"} | geography,acronym | geography,acronym |
| data from Hallegatte ( 2012 ) | {"year": "2012"} | {"geography": "LICs", "year": "2012"} | year | year,geography |
| national accounts | {"geography": "Turkey"} | {"geography": "Turkey"} | geography | geography |
| SRF registry of firms | {"acronym": "SRF", "geography": "Brazil", "producer": "Brazilian tax authority", "year": "2012"} | {"acronym": "SRF", "geography": "Brazil", "producer": "Brazilian tax authority", "year": "2012"} | producer,year,geography,acronym | producer,year,geography,acronym |
| Human Capital Index | {"geography": "Kenya", "producer": "World Bank", "year": "2020"} | {"geography": "Kenya", "producer": "World Bank", "year": "2020"} | producer,year,geography | producer,year,geography |
| United Nations population counts by age | {"producer": "United Nations"} | {"producer": "United Nations"} | producer | producer |
| 2002 Census | {"geography": "Rwanda", "year": "2002"} | {"geography": "Rwanda", "year": "2002"} | year,geography | year,geography |
| ENCASEH 1997 | {"acronym": "ENCASEH", "year": "1997"} | {"acronym": "ENCASEH", "year": "1997"} | year,acronym | year,acronym |
| Agricultural Wages in India | {"acronym": "AWI", "geography": "India", "producer": "Ministry of Agriculture"} | {"acronym": "AWI", "geography": "India", "producer": "Ministry of Agriculture"} | producer,geography,acronym | producer,geography,acronym |
| COVID-19 Enterprise Survey | {"geography": "Chad", "year": "2020"} | {"acronym": "ES", "geography": "Chad", "year": "2020"} | year,geography | year,geography,acronym |