NagaYu/customs-decomposer

Customs Span Tagger

A small word-level transformer that tags the personal spans of a request, so a device can decide which clauses may leave it. It is the learned detector inside the Customs research prototype's SemanticDecomposer: spans it finds are either replaced by a de-identified derived key or kept on-device entirely.

The interesting case it exists for is positional: in this world the same city string is personal when it is the user's home city and public when it is a travel destination. A gazetteer tags both. A personal-store lookup tags only the first, but only because it was handed the answer. This model has to read the sentence.

Architecture

architecture	customs-span-tagger
d_model	128
layers	2
attention heads	4
feed-forward width	256
max sequence length	128
vocabulary size	481
BIO tags	31

Word embeddings with learned positions, a pre-norm transformer encoder, and a linear head over 31 BIO tags derived from customs.types.DataClass. Unseen tokens fall back to a character-shape key (AC-19384756 -> <shape:A-d>), which is what gives the tagger recall on freshly drawn account numbers, device identifiers and phone numbers that could never have been memorised.

No pretrained checkpoint or tokenizer is downloaded; the vocabulary is built from the synthetic corpus alone.

Files

format	files
safetensors	model.safetensors
mlx	model.mlx.safetensors
gguf	model.gguf

config.json and vocab.json are required to load the model in any format -- the vocabulary carries the shape-backoff keys the tagger relies on.

Usage

from customs.model.tagger import SpanTagger

tagger = SpanTagger.from_pretrained("NagaYu/customs-decomposer")
for span in tagger.predict_spans("I live in Aveiro and I take Glucoform daily."):
    print(span.start, span.end, span.text, span.data_class.slug)

It satisfies the customs.detector.Detector protocol, so it drops straight into the decomposer:

from customs.decomposer import SemanticDecomposer
from customs.store import PersonalStore

store = PersonalStore(values={"HOME_CITY": "Aveiro", "MEDICATION": "Glucoform"})
decomposer = SemanticDecomposer(store, detector=tagger)
plan = decomposer.decompose("r-1", "I live in Aveiro and I take Glucoform. Max daily dose?")
print(plan.remote_payload_text())

Evaluation

Reported by the training run, and shipped verbatim as train_report.json:

metric	value
seed	0
steps	1,500
n_parameters	347,167
vocab_size	481
n_tags	31
seconds	33.9500
data_provenance	fully synthetic; generated by customs.world + customs.templates

corpus

metric	value
requests	1,440
profiles	60
train	1,080
val	360

train

metric	value
precision	1.0000
recall	1.0000
f1	1.0000
value_recall	1.0000
n_gold	2,430
n_pred	2,430

val

metric	value
precision	1.0000
recall	1.0000
f1	1.0000
value_recall	1.0000
n_gold	810
n_pred	810

held_out_check

metric	value
covered	72
total	72
rate	1.0000

Not printed here: ambiguity_probe, history. The full report ships as train_report.json in this repo.

Precision matters as much as recall here. A span wrongly tagged personal is a clause needlessly kept on-device, which costs answer quality, so span-F1 rather than recall is the target.

Training data

Trained entirely on the synthetic Customs benchmark, generated by scripts/build_bench.py from the invented tables in customs/world.py. Every name, city, employer, medication, condition, account identifier, email address, phone number and device identifier is fictional; the email domain used throughout is the reserved .invalid TLD. No real personal information was used at any stage, and no external dataset was downloaded.

Splits are cut by synthetic profile, not by request, so a tagger cannot score well at test time by memorising a training profile's employer string.

Scope of the claim

This model supports two specific, measurable claims and no others:

1. the volume of personal content crossing a trust boundary can be reduced, by detecting personal spans and emitting derived keys in their place; and
2. whatever does cross can be recorded, in the egress certificate the surrounding system writes.

Derived keys are themselves informative -- a country code or an age band is not nothing -- so this is data minimisation. It is not a claim of complete privacy, and nothing in this repository should be read as one. The safety property of the surrounding decomposer is deliberately not a function of this model being good: clauses containing a personal span with no derived stand-in are withheld regardless, and an independent egress guard re-scans the constructed payload against the personal store.

Limitations

• Trained on template-generated text. Surface diversity is far below real user writing, and the tag set is the closed taxonomy in customs/types.py.
• The vocabulary is corpus-specific. Out-of-world vocabulary reaches the model only through the shape backoff.
• The GGUF file is a container for this repository's custom span-tagger architecture, not a drop-in for a general-purpose runtime's built-in model graphs.
• Fictional-world performance is not evidence of real-world PII detection performance.

The project this belongs to

This tagger is one component of Customs, a research prototype for routing across an on-device / private-cloud / external-cloud model hierarchy when the middle tier is free but rationed daily. The tagger decides which spans are personal; the surrounding system decides what to do about it -- bind the private half on-device and send only a de-identified derived key, price the free ration as an online knapsack, record every crossing, and walk a degradation ladder instead of falling off a cliff when the ration runs out.

Full code, benchmark, figures and a Swift package: https://github.com/NagaYu/customs

Against confidence-threshold escalation on that benchmark (24 seeds), the full system delivered +6.0% whole-day answer quality, +52.9% evening quality, a 73% smaller quality gap across ration exhaustion, 14% less paid-tier spend, and zero personal tokens crossing any boundary instead of 49.5. Those numbers are a property of the whole system, not of this model alone.

Provenance

Exported from tagger by scripts/push_model.py via customs.model.export.export_all().

License

Apache-2.0.