Spaces:

ITKES
/

id-ocr-engine

Running

App Files Files Community

id-ocr-engine / README.md

Kevin-KES

Deploy current main to HF Space

50c6ee2 about 2 months ago

preview code

Raw

History Blame Contribute Delete

12.9 kB

metadata

title: ID OCR Engine
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860

ID OCR Engine

A Flask microservice that extracts identity fields from ID cards, passports, and product serial numbers using a multi-engine pipeline: VLM (Vision Language Model) + PaddleOCR + barcode/MRZ reading.

Highlights

Multi-engine ID extraction (barcode/MRZ + VLM + OCR) with cross-validation
Product serial number scanning via VLM with digit-density scoring
Live camera scanning with auto-detection (barcode + serial number modes)
Built-in web UI at / with file upload and camera support
JSON logs with request IDs
Rate limiting and optional API key auth
No persistent storage of uploaded images (files are deleted immediately after processing)
Models used:
- VLM: Qwen/Qwen3-VL-8B-Instruct via Hugging Face Inference API (configurable)
- OCR: PaddleOCR (English)
- MRZ: FastMRZ (ONNX)

Supported Document Types

Type	`doc_type`	Input	Machine-readable
SA Smart Card	`sa_id_card`	Front (required) + Back (optional, PDF417 barcode)	Barcode on back
SA Green Book	`sa_id_book`	Front (required)	None
Passport (any country)	`passport`	Front (required)	MRZ on front
Product Serial Number	`product_serial`	Product image	None

Extraction Pipelines

sa_id_card:      Barcode (back) → VLM (front) → PaddleOCR (front) → Cross-validate
sa_id_book:      VLM (front) → PaddleOCR (front) → Cross-validate
passport:        MRZ (front) → VLM (front) → PaddleOCR (front) → Cross-validate
product_serial:  VLM (center-cropped frame) → digit-density scoring → validation

Web UI

Extracts ID number, surname, names, date of birth, sex, country of birth, and citizenship status
Validates the 13-digit SA ID number using the SA-specific Luhn algorithm
Cross-validates OCR-extracted fields (DOB, gender, citizenship) against values encoded in the ID number
Multi-pass OCR with fallback preprocessing (CLAHE, adaptive thresholding, high-contrast) for difficult images
Automatic front/back card detection
Built-in web dashboard with drag-and-drop upload, batch processing, history, and CSV export
API key authentication, rate limiting, and request tracking
Automatic cleanup of uploaded images (personal data is never persisted)

Web Dashboard

Navigate to http://localhost:5001 to access the interactive dashboard:

Drag-and-drop or browse to upload ID card images
Batch processing queue for multiple images
Real-time results display with validation status
Processing history and statistics
CSV export of results

API

`GET /health`

Health check (no auth).

`GET /warm`

Warm up VLM + PaddleOCR + MRZ models (no auth). Call before first OCR request.

`POST /api/ocr`

Extract fields from an ID document image.

Headers:

Header	Required	Description
`X-API-Key`	Yes (if configured)	API authentication key
`X-Request-ID`	No	Client-provided request ID (echoed back)

Request: multipart/form-data

Field	Required	Description
`front`	Yes	Front image (or `file` for legacy)
`back`	No	Back image (only for `sa_id_card`)
`doc_type`	Yes	`sa_id_card`, `sa_id_book`, or `passport`

Accepted formats: png, jpg, jpeg, bmp, webp (max 10MB).

Response:

The response includes:

fields extracted from barcode/MRZ/VLM/OCR
checks for validation and cross-checks
confidence scores from OCR
quality metrics (brightness, blur, resolution)
raw_text OCR text

Example (SA ID):

{
  "doc_type": "sa_id_card",
  "fields": {
    "id_number": "8801015800082",
    "surname": "SMITH",
    "names": "JOHN PETER",
    "date_of_birth": "1988-01-01",
    "sex": "Male",
    "nationality": "South African",
    "country_of_birth": "RSA",
    "citizenship_status": "SA Citizen"
  },
  "barcode_data": { "...": "..." },
  "mrz_data": null,
  "extraction_method": "barcode+vlm",
  "checks": {
    "image_quality": "passed",
    "id_number_valid": "passed",
    "barcode_valid": "passed",
    "data_crosscheck": "passed",
    "dob_crosscheck": "passed",
    "gender_crosscheck": "passed",
    "mrz_valid": "not_applicable"
  },
  "overall_result": "pass"
}

Example (Passport):

{
  "doc_type": "passport",
  "fields": {
    "passport_number": "A12345678",
    "surname": "Smith",
    "given_names": "John Peter",
    "date_of_birth": "1988-01-01",
    "sex": "Male",
    "nationality": "ZAF",
    "expiry_date": "2030-01-01",
    "issuing_country": "ZAF",
    "id_number": "8801015800082"
  },
  "mrz_data": { "...": "..." },
  "extraction_method": "mrz+vlm",
  "checks": {
    "image_quality": "passed",
    "mrz_valid": "passed",
    "id_number_valid": "passed"
  },
  "overall_result": "pass"
}

`POST /api/serial`

Extract a product serial number from an image.

Request: multipart/form-data

Field	Required	Description
`image`	Yes	Product image file

Response:

{
  "serial_number": "WPHK002510002632",
  "bounding_box": null,
  "cropped_image": null,
  "extraction_source": "vlm",
  "confidence": "medium",
  "validation": {
    "valid": true,
    "normalized": "WPHK002510002632",
    "issues": [],
    "prefix_match": null
  },
  "failure_reason": null,
  "regions_detected": 0,
  "quality": { "width": 640, "height": 480, "usable": true, "issues": [] }
}

The serial extraction pipeline uses VLM to read all visible text, then scores candidates by digit density, rejects known non-serial patterns (IMEI, MAC, English phrases, sequential digits), and boosts values near S/N or Serial labels.

`POST /api/barcode/parse`

Parse raw barcode text (decoded client-side) into structured SA ID fields. Used by the web UI when the browser's BarcodeDetector API decodes a PDF417 barcode from the camera feed.

Request: application/json

{ "text": "<raw barcode data>" }

Response: Same format as /api/ocr with extraction_method: "barcode".

Checks and Quality

The checks object includes a per-check status: passed, failed, or not_applicable.

image_quality: minimum resolution check per doc type
id_number_valid: country-specific validation (Luhn for SA, regex for others)
mrz_valid: MRZ check digit validation
barcode_valid: barcode extraction succeeded
data_crosscheck: compare machine-readable data against VLM/OCR values
dob_crosscheck, gender_crosscheck: SA ID number-encoded validation

The quality object includes:

resolution_ok, width, height
brightness and blur_score
issues list for low-light, blur, or very small images

Country ID Validation

ID numbers are validated per detected nationality:

Country	Code	Format	Validation
South Africa	ZAF	13 digits	SA Luhn + encoded fields
Nigeria	NGA	11 digits	Regex
Kenya	KEN	1-9 digits	Regex
Zimbabwe	ZWE	8-9 digits + letter + 2 digits	Regex
Uganda	UGA	14 alphanumeric	Regex
Zambia	ZMB	10 digits	Regex

Configuration

Variable	Default	Description
`OCR_API_KEY`	(empty)	API key (disabled if empty)
`OCR_HOST`	`0.0.0.0`	Bind address
`OCR_PORT`	`7860`	Bind port
`OCR_DEBUG`	`false`	Enable Flask debug mode
`OCR_MAX_REQUEST_SIZE`	`10485760`	Max request size in bytes (default 10MB)
`OCR_UPLOAD_DIR`	`./uploads`	Temporary upload directory
`HF_API_TOKEN`	(empty)	HF Inference API token (for VLM)
`OCR_VLM_MODEL`	`Qwen/Qwen3-VL-8B-Instruct`	VLM model ID (for ID documents)
`OCR_SERIAL_VLM_MODEL`	`Qwen/Qwen3-VL-8B-Instruct`	VLM model ID (for serial numbers)
`OCR_LOG_LEVEL`	`INFO`	Logging level
`OCR_LOG_FORMAT`	`json`	Log format (`json` or `text`)
`OCR_MAX_REQUEST_SIZE`	`10485760`	Max upload size in bytes (default 10MB)
`OCR_UPLOAD_DIR`	`./uploads`	Temporary file storage directory
`OCR_WORKERS`	`min(CPU_count, 2)`	Gunicorn worker count

Running

Development:

python main.py

Production:

gunicorn -c gunicorn.conf.py wsgi:app

Docker Compose:

docker compose up --build

Docker:

docker build -t id-ocr .
docker run -p 7860:7860 -e OCR_API_KEY=your-key -e HF_API_TOKEN=hf_xxx id-ocr

Example Requests

# SA Smart Card (front only)
curl -X POST -H "X-API-Key: key" \
  -F "front=@front.jpg" -F "doc_type=sa_id_card" \
  http://localhost:7860/api/ocr

# SA Smart Card (front + back)
curl -X POST -H "X-API-Key: key" \
  -F "front=@front.jpg" -F "back=@back.jpg" -F "doc_type=sa_id_card" \
  http://localhost:7860/api/ocr

# Passport
curl -X POST -H "X-API-Key: key" \
  -F "front=@passport.jpg" -F "doc_type=passport" \
  http://localhost:7860/api/ocr

# Product serial number
curl -X POST -H "X-API-Key: key" \
  -F "image=@product.jpg" \
  http://localhost:7860/api/serial

Data Handling and Privacy

├── main.py                  # Flask app factory, routes, middleware
├── config.py                # Environment-based configuration
├── wsgi.py                  # Gunicorn WSGI entry point
├── gunicorn.conf.py         # Gunicorn worker configuration
├── logging_config.py        # Structured JSON logging
├── engine/
│   ├── ocr.py               # Image preprocessing + EasyOCR wrapper
│   └── id_parser.py         # SA ID parsing, Luhn validation, field extraction
├── static/
│   ├── index.html           # Web dashboard UI
│   ├── css/
│   │   └── style.css        # Dashboard styles
│   └── js/
│       ├── app.js           # App initialization
│       ├── api.js           # API client wrapper
│       ├── upload.js        # File upload & batch processing
│       ├── results.js       # Results rendering
│       ├── history.js       # Processing history management
│       ├── stats.js         # Statistics calculation
│       └── utils.js         # Utility functions
├── tests/
│   ├── test_api.py          # API integration tests
│   └── test_id_parser.py    # ID parser unit tests
├── Dockerfile
└── docker-compose.yaml

How It Works

OCR Pipeline

Image intake — validates format and size, saves to a temporary directory
Preprocessing — auto-brightness correction (gamma), CLAHE contrast enhancement, bilateral noise filtering, intelligent resizing (800–2000px width)
OCR pass 1 — standard preprocessing with EasyOCR
Fallback passes — if the ID number is not found, retries with aggressive preprocessing (sharpening + adaptive thresholding) and high-contrast preprocessing (histogram stretch + Otsu's thresholding)
Field extraction — spatial matching of bounding boxes to locate field values relative to labels
Validation — Luhn check on ID number, cross-validation of DOB/gender/citizenship against encoded values
Cleanup — temporary image is deleted immediately after processing

OCR Error Correction

Common character confusions are automatically corrected during ID number extraction (e.g., O→0, I→1, S→5).

SA ID Number Format

Logs can be JSON or text format.
X-Request-ID is accepted on requests and echoed in responses.
Each request is logged with latency and request ID.

Testing

python -m pytest tests/ -v