notebooks/MODEL_CARD.md

metadata

license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct-2507
datasets:
  - ricalanis/scrubdata-sft
tags:
  - data-cleaning
  - structured-output
  - json
  - tabular
  - gguf
  - llama.cpp
pipeline_tag: text-generation

ScrubData Planner — Qwen3-4B (QLoRA)

A ≤4B model fine-tuned to be a hands-off tabular data-cleaning planner: it reads a profile of a messy spreadsheet (per-column dtype, null/duplicate counts, detected semantic type, sample values) and emits a structured JSON cleaning plan. Deterministic pandas executes the plan — the model only plans. Built for the Build Small Hackathon (Backyard AI track), targeting Tiny Titan (≤4B) and Well-Tuned.

• Base: Qwen/Qwen3-4B-Instruct-2507 (4.0B, Apache-2.0)
• Method: QLoRA (Unsloth), r=32, 2 epochs, on an A100
• Data: ricalanis/scrubdata-sft — self-verified synthetic pairs (every example's plan was checked to recover the known-clean original by running the executor) backed by real vocabularies (countries/states/currencies/cities/industries/units) for genuine canonicalization, plus anomaly-flag and typo-cluster cases.
• GGUF: ricalanis/scrubdata-qwen3-4b-gguf (Q4_K_M, llama.cpp).

What it's for

Standardizing formats (dates/numbers/phones), canonicalizing inconsistent categories (USA/U.S.A/united states → United States), normalizing disguised nulls, de-duplicating, and flagging anomalies — with every change explained and reversible.

Evaluation

Scored on a frozen held-out gold set + a real OOD slice (Raha hospital). The fine-tune target is to clearly beat the rule-based heuristic, especially on alias-level canonicalization (the fuzzy skill rules can't do).

metric (synthetic, frozen gold)	heuristic	big vanilla (glm-5.1)	this 4B	oracle
json_valid	1.000	1.000	1.000	1.000
op_f1	0.961	0.891	0.998	1.000
canon_f1	0.133	0.452	0.864	1.000
recovery	0.627	0.747	0.932	1.000

Result: on its target distribution the 4B fine-tune beats a big generic model (canon_f1 0.45 → 0.86) and clears 3/4 goalposts (recovery 0.932 just under 0.95) — the small-aligned-model thesis, validated by measurement.

Known limitation (v1): it degenerates on very large/wide tables (1000×20 real benchmark) — trained only on small ones. Fixed in the v3 dataset (tables up to 90×9); retrain pending. Real Backyard-AI spreadsheets (dozens–hundreds of rows × a few columns) are within the trained range.

Usage (llama.cpp / Ollama)

ollama run hf.co/ricalanis/scrubdata-qwen3-4b-gguf

System prompt + profile→plan format: see scrubdata/prompt.py in the project repo.

Limitations

Plans only — it never edits data directly. Format standardization is opinionated (parses 100%→1.0, reformats phones); on datasets with different conventions this is a feature, not error-correction. Open-ended typo/entity-resolution beyond seen vocabulary is the remaining hard tail.