README.md

metadata

license: apache-2.0
pipeline_tag: text-generation
tags:
  - lfm2
  - data-use
  - provenance
  - usage
  - impact
  - sft
  - lora

lfm2.5-350M-datause-multitask

LoRA SFT of LiquidAI/LFM2.5-350M for data-mention provenance attributes (producer / year / geography / acronym) and usage/impact classification (data_type / usage_action / impact_label / usage_summary).

Training

• base model: LiquidAI/LFM2.5-350M
• dataset: rafmacalaba/data-use-sft
• epochs: 3
• learning rate: 0.0002
• LoRA: r=16 alpha=32 dropout=0.05
• completion-only masking (loss on assistant JSON turn)

real holdout

Holdout n=5464. Exact string match of each emitted attribute against the gold label.

attribute	tp	fp	fn	precision	recall	f0.5	f1
producer	1062	372	387	0.7406	0.7329	0.7390	0.7367
year	1008	214	342	0.8249	0.7467	0.8080	0.7838
geography	1548	399	479	0.7951	0.7637	0.7886	0.7791
acronym	1197	166	219	0.8782	0.8453	0.8714	0.8615
overall	4815	1151	1427	0.8071	0.7714	0.7997	0.7888

Usage/impact macro-F1 (per head):

• data_type: 0.7163
• usage_action: 0.5259
• impact_label: 0.4380
• usage_summary: mean_sim=0.6564 grounded_rate=0.9170

Verbatim rate (emitted values that are substrings of the context): 5950/5966 = 0.9973

synthetic holdout

Holdout n=1730. Exact string match of each emitted attribute against the gold label.

attribute	tp	fp	fn	precision	recall	f0.5	f1
producer	116	19	22	0.8593	0.8406	0.8555	0.8498
year	122	26	29	0.8243	0.8079	0.8210	0.8161
geography	163	36	40	0.8191	0.8030	0.8158	0.8109
acronym	62	14	30	0.8158	0.6739	0.7828	0.7381
overall	463	95	121	0.8297	0.7928	0.8221	0.8109

Usage/impact macro-F1 (per head):

• data_type: 0.8358
• usage_action: 0.6240
• impact_label: 0.5103
• usage_summary: mean_sim=0.5191 grounded_rate=0.5597

Verbatim rate (emitted values that are substrings of the context): 556/558 = 0.9964