schift-ko-pii-v6

34M Korean PII detector with dual-path LoRA — person + address + organization.

Designed for court judgment de-identification, administrative document masking, and general Korean PII detection. The model uses a dual-path architecture: person/address detection runs through the base encoder, while organization detection uses LoRA-adapted features. This prevents entity interference — adding organization doesn't degrade person/address accuracy.

Benchmark (benchmark_v3, 473 cases)

Entity	F1	P	R	Count
private_person	0.935	0.964	0.908	411
private_organization	0.820	0.791	0.852	182
private_address	0.926	0.917	0.936	141
private_phone	1.000	1.000	1.000	65
Overall	0.903	0.908	0.899	473

Compared to v5 (person + address only, F1=0.823 on benchmark_v2): v6 adds organization detection while maintaining person/address performance.

What it detects

Entity	Detected by	Examples
private_person	Model	김민수, 남궁혜진, Mike Johnson
private_address	Model	서울특별시 강남구 테헤란로 521
private_organization	Model (LoRA path)	주식회사 삼성전자, (주)휴먼웍스, 사단법인 한국법률구조공단
phone_number	Postprocess (regex)	010-1234-5678, 055-000-4484
resident_id	Postprocess (regex + checksum)	850205-1234567
account_number	Postprocess (regex)	M12345678, 52가1234
private_date	Postprocess (regex)	2024년 3월 15일
private_email	Postprocess (regex)	user@example.com

Installation

pip install schift-ko-pii

Usage

from schift_ko_pii import detect, mask, apply

# Detect all PII
spans = detect(
    "피고 주식회사 삼성전자(대표이사 김민수)는 "
    "서울특별시 강남구 테헤란로 521에서 원고에게 손해를 가하였다.",
    postprocess=True
)
for s in spans:
    print(f"  [{s['label']}] {s['text']} (score: {s['score']:.2f})")
# [private_organization] 삼성전자 (score: 0.94)
# [private_person] 김민수 (score: 0.98)
# [private_address] 서울특별시 강남구 테헤란로 521 (score: 0.97)

# Mask and review
result = mask(
    "원고 (주)휴먼웍스의 대표이사 강정석은 피고에 대하여 용역비 지급을 구한다.",
    postprocess=True
)
print(result["masked"])
# 원고 [기관1]의 대표이사 [사람1]은 피고에 대하여 용역비 지급을 구한다.

# Selective replacement
clean = apply(result["masked"], result["entities"], {
    "organization_1": "○○회사",
    "person_1": "○○○",
})

Architecture

• Encoder: LFM2.5 conv + attention hybrid, 6 layers, hidden 512 (frozen from v5)
• LoRA: rank-16 adapters on attention q/v projections (4 projections, 123K params)
• Heads: per-entity classifiers
  • Person/Address: Linear(512 → 5) — reads from base encoder
  • Organization: MLP(512 → 128 → 5) — reads from LoRA-adapted encoder
• Inference: dual-path forward — base path for person/address, LoRA path for organization, cross-head overlap resolution
• Total params: 34.1M (34.0M frozen + 0.1M trainable LoRA + heads)
• Format: safetensors float32 (130 MB)
• Tokenizer: klue/roberta-base, max 512 tokens
• Requires: trust_remote_code=True

Training Data

Source	Count	Entity
Assembly meeting transcripts	89,000	Person names (auto-labeled from speaker patterns)
KLUE NER (CC-BY-SA-4.0)	26,008	Person + Organization + Address
창녕군 administrative documents	10,000	Person + Organization + Address + Phone
Augmented / disambig / legal negative	2,152	Mixed

License

Schift License v2.0 — Apache 2.0 base with revenue threshold. Free for everyone under $10M annual revenue. Research, education, and non-profit always permitted.

Acknowledgments

Training data contributed by 지방자치혁신연구원 (창녕군 정보공개청구 데이터 기반 합성 데이터셋).

Citation

@software{schift_ko_pii_v6_2026,
  author = {Schift Inc.},
  title = {schift-ko-pii-v6: Dual-Path LoRA Korean PII Detection},
  year = {2026},
  url = {https://huggingface.co/schift-io/schift-ko-pii-v6},
}