examples/quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# document-pii-redactor \u2014 quickstart\n",
    "\n",
    "Detect PII in **document images** and **plain text**, then **redact**, **anonymize**, or **de-identify** it.\n",
    "\n",
    "This notebook walks through every transform on both modalities. Point `IMAGE` at any document image (a lab report, prescription, ID card, \u2026) and run top to bottom.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "%pip install -q \"document-pii-redactor[visual]\"   # [visual] adds signature/QR/face detection (AGPL-3.0)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tesseract is only needed for the built-in OCR (skip if you bring your own OCR):\n",
    "`apt-get install tesseract-ocr`  /  `brew install tesseract`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup \u2014 load once, detect once\n",
    "\n",
    "`detect()` is the core primitive: it runs OCR + the models a single time and returns structured entities (category, location, text, confidence). Every transform consumes its output.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from document_pii_redactor import ImagePIIRedactor, TextPIIRedactor\n",
    "\n",
    "IMAGE = \"report.png\"          # <- your document image\n",
    "\n",
    "image_redactor = ImagePIIRedactor(\"ekacare/document-pii-redactor\")\n",
    "entities = image_redactor.detect(IMAGE)\n",
    "\n",
    "for e in entities[:10]:\n",
    "    print(f\"{e.kind:6} {e.category:25} {str(e.text)[:40]!r}\")\n",
    "print(f\"\u2026 {len(entities)} entities total\")\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "text_redactor = TextPIIRedactor(\"ekacare/document-pii-redactor\")\n",
    "\n",
    "text = \"Mr. John Doe, 45 yrs, DOB 12-03-1979, Indiranagar, Bangalore. Contact: +91 98765 43210.\"\n",
    "spans = text_redactor.detect(text)\n",
    "\n",
    "for sp in spans:\n",
    "    print(f\"{sp.category:22} {sp.text!r}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Redact \u2014 destroy\n",
    "\n",
    "Black-out / blur / pixelate image regions; mask text spans. One-way, nothing kept.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "redacted = image_redactor.redact(IMAGE, entities, mode=\"blur\")   # or \"solid\" / \"pixelate\"\n",
    "redacted\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "text_redactor.redact(text, spans)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "text_redactor.redact(text, spans, mask=\"[{category}]\")   # mask can name the category\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anonymize \u2014 generalize\n",
    "\n",
    "One-way, no mapping kept. Ages become 10-year buckets, dates keep only the year, fine geography collapses to `[LOCATION]` (state and country survive), everything else becomes an unnumbered token.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "image_redactor.anonymize(IMAGE, entities)\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "text_redactor.anonymize(text, spans)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## De-identify \u2014 pseudonymize\n",
    "\n",
    "Every entity becomes a consistent pseudonym (`Person_1`) \u2014 same value, same pseudonym throughout the document \u2014 and the entity\u2192pseudonym mapping comes back so an authorized caller can re-link later.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "deid = image_redactor.deidentify(IMAGE, entities)\n",
    "deid.image\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "deid.mapping.entries\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "result = text_redactor.deidentify(text, spans)\n",
    "print(result.text)\n",
    "result.mapping.entries\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Processing multiple pages of the **same record**? Thread the mapping so numbering stays consistent:\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "page2 = \"Follow-up for Mr. John Doe. Contact +91 98765 43210.\"\n",
    "spans2 = text_redactor.detect(page2)\n",
    "text_redactor.deidentify(page2, spans2, mapping=result.mapping).text   # John Doe is still Person_1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hash tokens \u2014 stable across documents\n",
    "\n",
    "`strategy=\"hash\"` derives the pseudonym from the value itself, so the same value gets the same token in **every** document with no mapping to thread. `secret=` salts the hash so guessable values (names, phone numbers) can't be dictionary-reversed.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "text_redactor.deidentify(text, spans, strategy=\"hash\", secret=\"my-org-salt\").text\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bring your own OCR\n",
    "\n",
    "Prefer Textract / Google Vision / your own OCR over the built-in Tesseract? Pass the words with their **pixel-coordinate** boxes and OCR is skipped entirely \u2014 your exact boxes come back on the detected entities.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "entities_byo = image_redactor.detect(IMAGE,\n",
    "                                     words=[\"John\", \"Doe\"],\n",
    "                                     boxes=[[100, 20, 140, 40], [145, 20, 180, 40]])\n",
    "[(e.category, e.text, e.bbox) for e in entities_byo]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Limiting categories\n",
    "\n",
    "`categories=[...]` on `detect()` limits which of the 53 PII categories are found (default: all).\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "only_names = text_redactor.detect(text, categories=[\"primary_subject_name\", \"phone_mobile\"])\n",
    "text_redactor.redact(text, only_names)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**More**: [GitHub](https://github.com/eka-care/document-pii-redactor) \u00b7 [live demo](https://huggingface.co/spaces/ekacare/document-pii-redactor) \u00b7 [model weights](https://huggingface.co/ekacare/document-pii-redactor) \u00b7 [PyPI](https://pypi.org/project/document-pii-redactor/)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}