weft-lineage-extractor-0.5b — smallest scale point of a NEGATIVE RESULT

⚠️ RESEARCH ARTIFACT — the 0.5B point of a synthetic-only training study. Not a production tool.

✅ Resolved by real data: use weft-lineage-extractor-3b (real corpus, real precision 0.64). Full study: weft-lineage-extractor-1.5b.

The 0.5B point of a study showing that synthetic-only training induces a verbatim memorization leak in small models for ETL table-lineage extraction. It is the smallest scale point and shows the worst leak: near-perfect synthetic precision (0.994) collapses to 0.243 on real GitHub scripts, with 37.4% of hallucinations being table names recited verbatim from the synthetic training pool.

Same recipe as the 1.5B main model (LoRA on Qwen2.5-Coder-Instruct, Python/Shell synthetic ETL scripts, zero real scripts); only the base size differs.

This point's numbers (table-level, Convention A)

metric synthetic held-out real GitHub ETL
precision 0.994 0.243
direction accuracy 0.369
verbatim memorization leak 37.4%

Where it fits (scale curve)

scale real precision real direction verbatim leak
0.5B (this) 0.243 0.369 37.4%
1.5B (main) 0.270 0.496 22.4%
3B (synthetic) 0.325 0.468 10.9%
3B (real corpus) 0.64 ~0

Leak shrinks with scale (capacity), but only real training data closes the real-world gap (bottom row). Direction confusion does not improve with size.

Intended use

  • ✅ Reproducing / studying the synthetic-training memorization-leak failure at minimal scale.
  • ❌ Not for production lineage — use the real-corpus 3B.

Usage, prompt format, training details, citation

Identical to the main model (this variant uses task_type: PYTHON | SHELL; everything else the same): weft-lineage-extractor-1.5b.

Downloads last month
491
Safetensors
Model size
0.5B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for wallfacers/weft-lineage-extractor-0.5b

Adapter
(50)
this model

Dataset used to train wallfacers/weft-lineage-extractor-0.5b

Evaluation results

  • Table precision (synthetic held-out) on synthetic held-out (structural-form isolated)
    self-reported
    0.994
  • Table precision (real, out-of-distribution) on real GitHub ETL (human gold)
    self-reported
    0.243