File size: 2,185 Bytes
39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 51c043a 39b7684 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
license: apache-2.0
pipeline_tag: text-generation
tags:
- lfm2
- data-use
- provenance
- usage
- impact
- sft
- lora
---
# lfm2.5-350M-multitask-datause
LoRA SFT of `LiquidAI/LFM2.5-350M` for data-mention provenance attributes
(producer / year / geography / acronym) and usage/impact classification
(data_type / usage_action / impact_label / usage_summary).
## Training
- base model: `LiquidAI/LFM2.5-350M`
- dataset: `rafmacalaba/data-use-sft`
- epochs: 5
- learning rate: 0.0002
- LoRA: r=16 alpha=32 dropout=0.05
- completion-only masking (loss on assistant JSON turn)
## real holdout
Holdout n=14776. Exact string match of each emitted attribute against the gold label.
| attribute | tp | fp | fn | precision | recall | f0.5 | f1 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| producer | 1909 | 968 | 914 | 0.6635 | 0.6762 | 0.6660 | 0.6698 |
| year | 2264 | 520 | 631 | 0.8132 | 0.7820 | 0.8068 | 0.7973 |
| geography | 2632 | 984 | 968 | 0.7279 | 0.7311 | 0.7285 | 0.7295 |
| acronym | 1973 | 456 | 449 | 0.8123 | 0.8146 | 0.8127 | 0.8134 |
| **overall** | 8778 | 2928 | 2962 | 0.7499 | 0.7477 | 0.7494 | 0.7488 |
Usage/impact macro-F1 (per head):
- data_type: 0.7005
- usage_action: 0.6554
- impact_label: 0.5842
- usage_summary: mean_sim=0.5522 grounded_rate=0.6513
Verbatim rate (emitted values that are substrings of the context): 11659/11706 = 0.9960
## synthetic holdout
Holdout n=2305. Exact string match of each emitted attribute against the gold label.
| attribute | tp | fp | fn | precision | recall | f0.5 | f1 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| producer | 231 | 39 | 56 | 0.8556 | 0.8049 | 0.8449 | 0.8294 |
| year | 265 | 50 | 40 | 0.8413 | 0.8689 | 0.8466 | 0.8548 |
| geography | 344 | 63 | 53 | 0.8452 | 0.8665 | 0.8494 | 0.8557 |
| acronym | 145 | 84 | 36 | 0.6332 | 0.8011 | 0.6609 | 0.7073 |
| **overall** | 985 | 236 | 185 | 0.8067 | 0.8419 | 0.8135 | 0.8239 |
Usage/impact macro-F1 (per head):
- data_type: 0.7818
- usage_action: 0.6489
- impact_label: 0.5147
- usage_summary: mean_sim=0.5752 grounded_rate=0.6768
Verbatim rate (emitted values that are substrings of the context): 1216/1221 = 0.9959
|