docs/CHANGELOG.md

Changelog

Big changes since the fork of NoteGuard/Automatic-PII-preprocessing-tool, grouped by pull request / milestone (newest first). Encode Vibe Coding Hackathon — FLock Sovereign AI Challenge (hosted by Encode Hub).

Live-demo polish — 2026-06-21

Direct UI / docs changes on main for the public demo.

• Download de-identified output from the Try-it tab: a single sanitised note or the full sanitised dataset, as JSON or CSV — only the de-identified text is exported (raw PHI never leaves).
• Removed FLock.io as a technical reference throughout (UI + docs); kept only the hackathon credit.
• Added a hackathon footer to every tab.
• Simplified the UI copy: dropped model / algorithm names and file / command references; renamed the "Metrics & Leakage" tab; 1-based note index; 50 sample notes in the browser.
• Fixed the Streamlit outputs/ path after the app moved to the repo root (the Metrics and Two-Trust "Run" buttons read / wrote the wrong location).

[0.0.1] — 2026-06-20 — first release (Gold-RAP restructure & Hugging Face deploy)

Branches dev/refactor-cleancode + feat/hf-spaces-demo, merged to main.

• Reorganised to a Gold-RAP "analysis as a product" layout: src/ package, tests/, docs/, data/ (inputs), outputs/ (generated artifacts).
• Renamed package noteguard/ → src/; British spelling throughout (recognisers).
• Packaged via pyproject.toml as the single dependency source (removed requirements.txt).
• CI (.github/workflows/ci.yml) runs ruff + pytest on every push/PR; added logging.
• Streamlit demo moved to repo root (streamlit_app.py); run_eval.py moved under tests/.
• Hugging Face Spaces (Docker SDK) deploy: Dockerfile, README front-matter, docs/DEPLOY_HF_SPACES.md.
• Decluttered Claude tooling (eval skill, post-edit hook, local settings) and the root changelog.

Post-PR #3 hardening — 2026-06-20

Direct fixes on main after the team-briefing work.

• Only date-of-birth is masked; visit / admission dates are kept (clinically useful). Human-readable redaction labels (e.g. [NHS number]).
• Fixed pseudonym mis-mapping caused by spaCy ORGANIZATION over-tagging.
• Removed the roster / gazetteer feature (kept the metric honest — no circularity).
• Consolidated the frontend on Streamlit; removed the Gradio demo.

PR #3 — Action items 1–4 from the team briefing — 2026-06-20

• Default spaCy model en_core_web_lg (best UK-name recall; clinical transformer tested and rejected).
• Human-in-the-loop review queue: low-confidence spans are still redacted and flagged for IG analyst confirmation before SDE-pool admission.
• ORGANIZATION excluded from Presidio's kept entities (it over-tagged labels like "NHS"/"GMC"); NHS site names caught by a LOCATION rule instead.
• Overlap-safe, priority-ranked span merge (precise rule entities beat broad NER spans).

PR #2 — UK entity backport — 2026-06-20

• Added UK_NINO, UK_PASSPORT, UK_VEHICLE_REGISTRATION recognisers and comma-separated NHS-number detection, with format-correct fake surrogates for each.

PR #1 — NoteGuard de-identification gate — 2026-06-20

Consolidated two parallel implementations into one package.

• One pluggable Detector interface: RuleDetector / PresidioDetector, degrading gracefully to pure-Python rules when spaCy/Presidio are unavailable.
• Patient-consistent pseudonymisation via a Faker(en_GB) vault + per-patient date-shift; redaction mode.
• Evaluation harness: per-entity precision / recall / F1 and a measured residual-leakage rate, using the dataset's structured patient/admission tables as free ground truth.
• Two-Trust "sanitise-at-source" demo; Streamlit demo UI.

Initial NoteGuard implementation

• Pure-Python noteguard package: rule recognisers (NHS Modulus-11 + context anchor, postcode, date, phone, email), detection layer, redaction/pseudonymisation transforms, evaluation, pipeline.
• Loads the NHSE synthetic clinical notes (3 CSVs joined on person_id/admission_id).

Initial commit

• Fork of NoteGuard/Automatic-PII-preprocessing-tool.