README.md

---
title: ID OCR Engine
emoji: "\U0001F50D"
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
---

<p align="center">
  <img src="https://img.shields.io/badge/python-3.12+-3776AB?style=flat&logo=python&logoColor=white" alt="Python 3.12+">
  <img src="https://img.shields.io/badge/flask-3.1-000000?style=flat&logo=flask&logoColor=white" alt="Flask 3.1">
  <img src="https://img.shields.io/badge/docker-ready-2496ED?style=flat&logo=docker&logoColor=white" alt="Docker Ready">
  <img src="https://img.shields.io/badge/ocr-paddleocr-0A7D3B?style=flat&logo=python&logoColor=white" alt="PaddleOCR">
  <img src="https://img.shields.io/badge/mrz-fastmrz-4A5568?style=flat" alt="FastMRZ">
  <img src="https://img.shields.io/badge/barcode-zxing--cpp-6A1B9A?style=flat" alt="zxing-cpp">
</p>
<p align="center">
  <img src="https://img.shields.io/badge/api-rest-0B7285?style=flat" alt="REST API">
  <img src="https://img.shields.io/badge/logging-json-2B8A3E?style=flat" alt="JSON Logging">
  <img src="https://img.shields.io/badge/rate--limit-flask--limiter-364FC7?style=flat" alt="Rate limiting">
  <img src="https://img.shields.io/badge/security-api%20key-5C7CFA?style=flat" alt="API Key Auth">
  <img src="https://img.shields.io/badge/license-proprietary-8D99AE?style=flat" alt="License">
</p>

# ID OCR Engine

A Flask microservice that extracts identity fields from ID cards, passports, and product serial numbers using a multi-engine pipeline: VLM (Vision Language Model) + PaddleOCR + barcode/MRZ reading.

## Highlights

- Multi-engine ID extraction (barcode/MRZ + VLM + OCR) with cross-validation
- Product serial number scanning via VLM with digit-density scoring
- Live camera scanning with auto-detection (barcode + serial number modes)
- Built-in web UI at `/` with file upload and camera support
- JSON logs with request IDs
- Rate limiting and optional API key auth
- No persistent storage of uploaded images (files are deleted immediately after processing)
- Models used:
  - VLM: `Qwen/Qwen3-VL-8B-Instruct` via Hugging Face Inference API (configurable)
  - OCR: PaddleOCR (English)
  - MRZ: FastMRZ (ONNX)

## Supported Document Types

| Type | `doc_type` | Input | Machine-readable |
|------|-----------|-------|-----------------|
| SA Smart Card | `sa_id_card` | Front (required) + Back (optional, PDF417 barcode) | Barcode on back |
| SA Green Book | `sa_id_book` | Front (required) | None |
| Passport (any country) | `passport` | Front (required) | MRZ on front |
| Product Serial Number | `product_serial` | Product image | None |

## Extraction Pipelines

```
sa_id_card:      Barcode (back) → VLM (front) → PaddleOCR (front) → Cross-validate
sa_id_book:      VLM (front) → PaddleOCR (front) → Cross-validate
passport:        MRZ (front) → VLM (front) → PaddleOCR (front) → Cross-validate
product_serial:  VLM (center-cropped frame) → digit-density scoring → validation
```

## Web UI

- Extracts ID number, surname, names, date of birth, sex, country of birth, and citizenship status
- Validates the 13-digit SA ID number using the SA-specific Luhn algorithm
- Cross-validates OCR-extracted fields (DOB, gender, citizenship) against values encoded in the ID number
- Multi-pass OCR with fallback preprocessing (CLAHE, adaptive thresholding, high-contrast) for difficult images
- Automatic front/back card detection
- Built-in web dashboard with drag-and-drop upload, batch processing, history, and CSV export
- API key authentication, rate limiting, and request tracking
- Automatic cleanup of uploaded images (personal data is never persisted)

## Web Dashboard

Navigate to `http://localhost:5001` to access the interactive dashboard:

- Drag-and-drop or browse to upload ID card images
- Batch processing queue for multiple images
- Real-time results display with validation status
- Processing history and statistics
- CSV export of results

## API

### `GET /health`
Health check (no auth).

### `GET /warm`
Warm up VLM + PaddleOCR + MRZ models (no auth). Call before first OCR request.

### `POST /api/ocr`

Extract fields from an ID document image.

**Headers:**

| Header | Required | Description |
|---|---|---|
| `X-API-Key` | Yes (if configured) | API authentication key |
| `X-Request-ID` | No | Client-provided request ID (echoed back) |

**Request:** `multipart/form-data`

| Field | Required | Description |
|---|---|---|
| `front` | Yes | Front image (or `file` for legacy) |
| `back` | No | Back image (only for `sa_id_card`) |
| `doc_type` | Yes | `sa_id_card`, `sa_id_book`, or `passport` |

Accepted formats: `png`, `jpg`, `jpeg`, `bmp`, `webp` (max 10MB).

**Response:**

The response includes:

- `fields` extracted from barcode/MRZ/VLM/OCR
- `checks` for validation and cross-checks
- `confidence` scores from OCR
- `quality` metrics (brightness, blur, resolution)
- `raw_text` OCR text

**Example (SA ID):**

```json
{
  "doc_type": "sa_id_card",
  "fields": {
    "id_number": "8801015800082",
    "surname": "SMITH",
    "names": "JOHN PETER",
    "date_of_birth": "1988-01-01",
    "sex": "Male",
    "nationality": "South African",
    "country_of_birth": "RSA",
    "citizenship_status": "SA Citizen"
  },
  "barcode_data": { "...": "..." },
  "mrz_data": null,
  "extraction_method": "barcode+vlm",
  "checks": {
    "image_quality": "passed",
    "id_number_valid": "passed",
    "barcode_valid": "passed",
    "data_crosscheck": "passed",
    "dob_crosscheck": "passed",
    "gender_crosscheck": "passed",
    "mrz_valid": "not_applicable"
  },
  "overall_result": "pass"
}
```

**Example (Passport):**

```json
{
  "doc_type": "passport",
  "fields": {
    "passport_number": "A12345678",
    "surname": "Smith",
    "given_names": "John Peter",
    "date_of_birth": "1988-01-01",
    "sex": "Male",
    "nationality": "ZAF",
    "expiry_date": "2030-01-01",
    "issuing_country": "ZAF",
    "id_number": "8801015800082"
  },
  "mrz_data": { "...": "..." },
  "extraction_method": "mrz+vlm",
  "checks": {
    "image_quality": "passed",
    "mrz_valid": "passed",
    "id_number_valid": "passed"
  },
  "overall_result": "pass"
}
```

### `POST /api/serial`

Extract a product serial number from an image.

**Request:** `multipart/form-data`

| Field | Required | Description |
|---|---|---|
| `image` | Yes | Product image file |

**Response:**

```json
{
  "serial_number": "WPHK002510002632",
  "bounding_box": null,
  "cropped_image": null,
  "extraction_source": "vlm",
  "confidence": "medium",
  "validation": {
    "valid": true,
    "normalized": "WPHK002510002632",
    "issues": [],
    "prefix_match": null
  },
  "failure_reason": null,
  "regions_detected": 0,
  "quality": { "width": 640, "height": 480, "usable": true, "issues": [] }
}
```

The serial extraction pipeline uses VLM to read all visible text, then scores candidates by digit density, rejects known non-serial patterns (IMEI, MAC, English phrases, sequential digits), and boosts values near `S/N` or `Serial` labels.

### `POST /api/barcode/parse`

Parse raw barcode text (decoded client-side) into structured SA ID fields. Used by the web UI when the browser's BarcodeDetector API decodes a PDF417 barcode from the camera feed.

**Request:** `application/json`

```json
{ "text": "<raw barcode data>" }
```

**Response:** Same format as `/api/ocr` with `extraction_method: "barcode"`.

## Checks and Quality

The `checks` object includes a per-check status: `passed`, `failed`, or `not_applicable`.

- `image_quality`: minimum resolution check per doc type
- `id_number_valid`: country-specific validation (Luhn for SA, regex for others)
- `mrz_valid`: MRZ check digit validation
- `barcode_valid`: barcode extraction succeeded
- `data_crosscheck`: compare machine-readable data against VLM/OCR values
- `dob_crosscheck`, `gender_crosscheck`: SA ID number-encoded validation

The `quality` object includes:

- `resolution_ok`, `width`, `height`
- `brightness` and `blur_score`
- `issues` list for low-light, blur, or very small images

## Country ID Validation

ID numbers are validated per detected nationality:

| Country | Code | Format | Validation |
|---------|------|--------|-----------|
| South Africa | ZAF | 13 digits | SA Luhn + encoded fields |
| Nigeria | NGA | 11 digits | Regex |
| Kenya | KEN | 1-9 digits | Regex |
| Zimbabwe | ZWE | 8-9 digits + letter + 2 digits | Regex |
| Uganda | UGA | 14 alphanumeric | Regex |
| Zambia | ZMB | 10 digits | Regex |

## Configuration

| Variable | Default | Description |
|---|---|---|
| `OCR_API_KEY` | *(empty)* | API key (disabled if empty) |
| `OCR_HOST` | `0.0.0.0` | Bind address |
| `OCR_PORT` | `7860` | Bind port |
| `OCR_DEBUG` | `false` | Enable Flask debug mode |
| `OCR_MAX_REQUEST_SIZE` | `10485760` | Max request size in bytes (default 10MB) |
| `OCR_UPLOAD_DIR` | `./uploads` | Temporary upload directory |
| `HF_API_TOKEN` | *(empty)* | HF Inference API token (for VLM) |
| `OCR_VLM_MODEL` | `Qwen/Qwen3-VL-8B-Instruct` | VLM model ID (for ID documents) |
| `OCR_SERIAL_VLM_MODEL` | `Qwen/Qwen3-VL-8B-Instruct` | VLM model ID (for serial numbers) |
| `OCR_LOG_LEVEL` | `INFO` | Logging level |
| `OCR_LOG_FORMAT` | `json` | Log format (`json` or `text`) |
| `OCR_MAX_REQUEST_SIZE` | `10485760` | Max upload size in bytes (default 10MB) |
| `OCR_UPLOAD_DIR` | `./uploads` | Temporary file storage directory |
| `OCR_WORKERS` | `min(CPU_count, 2)` | Gunicorn worker count |

## Running

**Development:**
```bash
python main.py
```

**Production:**
```bash
gunicorn -c gunicorn.conf.py wsgi:app
```

**Docker Compose:**
```bash
docker compose up --build
```

**Docker:**
```bash
docker build -t id-ocr .
docker run -p 7860:7860 -e OCR_API_KEY=your-key -e HF_API_TOKEN=hf_xxx id-ocr
```

## Example Requests

```bash
# SA Smart Card (front only)
curl -X POST -H "X-API-Key: key" \
  -F "front=@front.jpg" -F "doc_type=sa_id_card" \
  http://localhost:7860/api/ocr

# SA Smart Card (front + back)
curl -X POST -H "X-API-Key: key" \
  -F "front=@front.jpg" -F "back=@back.jpg" -F "doc_type=sa_id_card" \
  http://localhost:7860/api/ocr

# Passport
curl -X POST -H "X-API-Key: key" \
  -F "front=@passport.jpg" -F "doc_type=passport" \
  http://localhost:7860/api/ocr

# Product serial number
curl -X POST -H "X-API-Key: key" \
  -F "image=@product.jpg" \
  http://localhost:7860/api/serial
```

## Data Handling and Privacy

```
├── main.py                  # Flask app factory, routes, middleware
├── config.py                # Environment-based configuration
├── wsgi.py                  # Gunicorn WSGI entry point
├── gunicorn.conf.py         # Gunicorn worker configuration
├── logging_config.py        # Structured JSON logging
├── engine/
│   ├── ocr.py               # Image preprocessing + EasyOCR wrapper
│   └── id_parser.py         # SA ID parsing, Luhn validation, field extraction
├── static/
│   ├── index.html           # Web dashboard UI
│   ├── css/
│   │   └── style.css        # Dashboard styles
│   └── js/
│       ├── app.js           # App initialization
│       ├── api.js           # API client wrapper
│       ├── upload.js        # File upload & batch processing
│       ├── results.js       # Results rendering
│       ├── history.js       # Processing history management
│       ├── stats.js         # Statistics calculation
│       └── utils.js         # Utility functions
├── tests/
│   ├── test_api.py          # API integration tests
│   └── test_id_parser.py    # ID parser unit tests
├── Dockerfile
└── docker-compose.yaml
```

## How It Works

### OCR Pipeline

1. **Image intake** — validates format and size, saves to a temporary directory
2. **Preprocessing** — auto-brightness correction (gamma), CLAHE contrast enhancement, bilateral noise filtering, intelligent resizing (800–2000px width)
3. **OCR pass 1** — standard preprocessing with EasyOCR
4. **Fallback passes** — if the ID number is not found, retries with aggressive preprocessing (sharpening + adaptive thresholding) and high-contrast preprocessing (histogram stretch + Otsu's thresholding)
5. **Field extraction** — spatial matching of bounding boxes to locate field values relative to labels
6. **Validation** — Luhn check on ID number, cross-validation of DOB/gender/citizenship against encoded values
7. **Cleanup** — temporary image is deleted immediately after processing

### OCR Error Correction

Common character confusions are automatically corrected during ID number extraction (e.g., `O`→`0`, `I`→`1`, `S`→`5`).

## SA ID Number Format

- Logs can be JSON or text format.
- `X-Request-ID` is accepted on requests and echoed in responses.
- Each request is logged with latency and request ID.

## Testing

```bash
python -m pytest tests/ -v
```