---
title: ID OCR Engine
emoji: "\U0001F50D"
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
---
# ID OCR Engine
A Flask microservice that extracts identity fields from ID cards, passports, and product serial numbers using a multi-engine pipeline: VLM (Vision Language Model) + PaddleOCR + barcode/MRZ reading.
## Highlights
- Multi-engine ID extraction (barcode/MRZ + VLM + OCR) with cross-validation
- Product serial number scanning via VLM with digit-density scoring
- Live camera scanning with auto-detection (barcode + serial number modes)
- Built-in web UI at `/` with file upload and camera support
- JSON logs with request IDs
- Rate limiting and optional API key auth
- No persistent storage of uploaded images (files are deleted immediately after processing)
- Models used:
- VLM: `Qwen/Qwen3-VL-8B-Instruct` via Hugging Face Inference API (configurable)
- OCR: PaddleOCR (English)
- MRZ: FastMRZ (ONNX)
## Supported Document Types
| Type | `doc_type` | Input | Machine-readable |
|------|-----------|-------|-----------------|
| SA Smart Card | `sa_id_card` | Front (required) + Back (optional, PDF417 barcode) | Barcode on back |
| SA Green Book | `sa_id_book` | Front (required) | None |
| Passport (any country) | `passport` | Front (required) | MRZ on front |
| Product Serial Number | `product_serial` | Product image | None |
## Extraction Pipelines
```
sa_id_card: Barcode (back) → VLM (front) → PaddleOCR (front) → Cross-validate
sa_id_book: VLM (front) → PaddleOCR (front) → Cross-validate
passport: MRZ (front) → VLM (front) → PaddleOCR (front) → Cross-validate
product_serial: VLM (center-cropped frame) → digit-density scoring → validation
```
## Web UI
- Extracts ID number, surname, names, date of birth, sex, country of birth, and citizenship status
- Validates the 13-digit SA ID number using the SA-specific Luhn algorithm
- Cross-validates OCR-extracted fields (DOB, gender, citizenship) against values encoded in the ID number
- Multi-pass OCR with fallback preprocessing (CLAHE, adaptive thresholding, high-contrast) for difficult images
- Automatic front/back card detection
- Built-in web dashboard with drag-and-drop upload, batch processing, history, and CSV export
- API key authentication, rate limiting, and request tracking
- Automatic cleanup of uploaded images (personal data is never persisted)
## Web Dashboard
Navigate to `http://localhost:5001` to access the interactive dashboard:
- Drag-and-drop or browse to upload ID card images
- Batch processing queue for multiple images
- Real-time results display with validation status
- Processing history and statistics
- CSV export of results
## API
### `GET /health`
Health check (no auth).
### `GET /warm`
Warm up VLM + PaddleOCR + MRZ models (no auth). Call before first OCR request.
### `POST /api/ocr`
Extract fields from an ID document image.
**Headers:**
| Header | Required | Description |
|---|---|---|
| `X-API-Key` | Yes (if configured) | API authentication key |
| `X-Request-ID` | No | Client-provided request ID (echoed back) |
**Request:** `multipart/form-data`
| Field | Required | Description |
|---|---|---|
| `front` | Yes | Front image (or `file` for legacy) |
| `back` | No | Back image (only for `sa_id_card`) |
| `doc_type` | Yes | `sa_id_card`, `sa_id_book`, or `passport` |
Accepted formats: `png`, `jpg`, `jpeg`, `bmp`, `webp` (max 10MB).
**Response:**
The response includes:
- `fields` extracted from barcode/MRZ/VLM/OCR
- `checks` for validation and cross-checks
- `confidence` scores from OCR
- `quality` metrics (brightness, blur, resolution)
- `raw_text` OCR text
**Example (SA ID):**
```json
{
"doc_type": "sa_id_card",
"fields": {
"id_number": "8801015800082",
"surname": "SMITH",
"names": "JOHN PETER",
"date_of_birth": "1988-01-01",
"sex": "Male",
"nationality": "South African",
"country_of_birth": "RSA",
"citizenship_status": "SA Citizen"
},
"barcode_data": { "...": "..." },
"mrz_data": null,
"extraction_method": "barcode+vlm",
"checks": {
"image_quality": "passed",
"id_number_valid": "passed",
"barcode_valid": "passed",
"data_crosscheck": "passed",
"dob_crosscheck": "passed",
"gender_crosscheck": "passed",
"mrz_valid": "not_applicable"
},
"overall_result": "pass"
}
```
**Example (Passport):**
```json
{
"doc_type": "passport",
"fields": {
"passport_number": "A12345678",
"surname": "Smith",
"given_names": "John Peter",
"date_of_birth": "1988-01-01",
"sex": "Male",
"nationality": "ZAF",
"expiry_date": "2030-01-01",
"issuing_country": "ZAF",
"id_number": "8801015800082"
},
"mrz_data": { "...": "..." },
"extraction_method": "mrz+vlm",
"checks": {
"image_quality": "passed",
"mrz_valid": "passed",
"id_number_valid": "passed"
},
"overall_result": "pass"
}
```
### `POST /api/serial`
Extract a product serial number from an image.
**Request:** `multipart/form-data`
| Field | Required | Description |
|---|---|---|
| `image` | Yes | Product image file |
**Response:**
```json
{
"serial_number": "WPHK002510002632",
"bounding_box": null,
"cropped_image": null,
"extraction_source": "vlm",
"confidence": "medium",
"validation": {
"valid": true,
"normalized": "WPHK002510002632",
"issues": [],
"prefix_match": null
},
"failure_reason": null,
"regions_detected": 0,
"quality": { "width": 640, "height": 480, "usable": true, "issues": [] }
}
```
The serial extraction pipeline uses VLM to read all visible text, then scores candidates by digit density, rejects known non-serial patterns (IMEI, MAC, English phrases, sequential digits), and boosts values near `S/N` or `Serial` labels.
### `POST /api/barcode/parse`
Parse raw barcode text (decoded client-side) into structured SA ID fields. Used by the web UI when the browser's BarcodeDetector API decodes a PDF417 barcode from the camera feed.
**Request:** `application/json`
```json
{ "text": "" }
```
**Response:** Same format as `/api/ocr` with `extraction_method: "barcode"`.
## Checks and Quality
The `checks` object includes a per-check status: `passed`, `failed`, or `not_applicable`.
- `image_quality`: minimum resolution check per doc type
- `id_number_valid`: country-specific validation (Luhn for SA, regex for others)
- `mrz_valid`: MRZ check digit validation
- `barcode_valid`: barcode extraction succeeded
- `data_crosscheck`: compare machine-readable data against VLM/OCR values
- `dob_crosscheck`, `gender_crosscheck`: SA ID number-encoded validation
The `quality` object includes:
- `resolution_ok`, `width`, `height`
- `brightness` and `blur_score`
- `issues` list for low-light, blur, or very small images
## Country ID Validation
ID numbers are validated per detected nationality:
| Country | Code | Format | Validation |
|---------|------|--------|-----------|
| South Africa | ZAF | 13 digits | SA Luhn + encoded fields |
| Nigeria | NGA | 11 digits | Regex |
| Kenya | KEN | 1-9 digits | Regex |
| Zimbabwe | ZWE | 8-9 digits + letter + 2 digits | Regex |
| Uganda | UGA | 14 alphanumeric | Regex |
| Zambia | ZMB | 10 digits | Regex |
## Configuration
| Variable | Default | Description |
|---|---|---|
| `OCR_API_KEY` | *(empty)* | API key (disabled if empty) |
| `OCR_HOST` | `0.0.0.0` | Bind address |
| `OCR_PORT` | `7860` | Bind port |
| `OCR_DEBUG` | `false` | Enable Flask debug mode |
| `OCR_MAX_REQUEST_SIZE` | `10485760` | Max request size in bytes (default 10MB) |
| `OCR_UPLOAD_DIR` | `./uploads` | Temporary upload directory |
| `HF_API_TOKEN` | *(empty)* | HF Inference API token (for VLM) |
| `OCR_VLM_MODEL` | `Qwen/Qwen3-VL-8B-Instruct` | VLM model ID (for ID documents) |
| `OCR_SERIAL_VLM_MODEL` | `Qwen/Qwen3-VL-8B-Instruct` | VLM model ID (for serial numbers) |
| `OCR_LOG_LEVEL` | `INFO` | Logging level |
| `OCR_LOG_FORMAT` | `json` | Log format (`json` or `text`) |
| `OCR_MAX_REQUEST_SIZE` | `10485760` | Max upload size in bytes (default 10MB) |
| `OCR_UPLOAD_DIR` | `./uploads` | Temporary file storage directory |
| `OCR_WORKERS` | `min(CPU_count, 2)` | Gunicorn worker count |
## Running
**Development:**
```bash
python main.py
```
**Production:**
```bash
gunicorn -c gunicorn.conf.py wsgi:app
```
**Docker Compose:**
```bash
docker compose up --build
```
**Docker:**
```bash
docker build -t id-ocr .
docker run -p 7860:7860 -e OCR_API_KEY=your-key -e HF_API_TOKEN=hf_xxx id-ocr
```
## Example Requests
```bash
# SA Smart Card (front only)
curl -X POST -H "X-API-Key: key" \
-F "front=@front.jpg" -F "doc_type=sa_id_card" \
http://localhost:7860/api/ocr
# SA Smart Card (front + back)
curl -X POST -H "X-API-Key: key" \
-F "front=@front.jpg" -F "back=@back.jpg" -F "doc_type=sa_id_card" \
http://localhost:7860/api/ocr
# Passport
curl -X POST -H "X-API-Key: key" \
-F "front=@passport.jpg" -F "doc_type=passport" \
http://localhost:7860/api/ocr
# Product serial number
curl -X POST -H "X-API-Key: key" \
-F "image=@product.jpg" \
http://localhost:7860/api/serial
```
## Data Handling and Privacy
```
├── main.py # Flask app factory, routes, middleware
├── config.py # Environment-based configuration
├── wsgi.py # Gunicorn WSGI entry point
├── gunicorn.conf.py # Gunicorn worker configuration
├── logging_config.py # Structured JSON logging
├── engine/
│ ├── ocr.py # Image preprocessing + EasyOCR wrapper
│ └── id_parser.py # SA ID parsing, Luhn validation, field extraction
├── static/
│ ├── index.html # Web dashboard UI
│ ├── css/
│ │ └── style.css # Dashboard styles
│ └── js/
│ ├── app.js # App initialization
│ ├── api.js # API client wrapper
│ ├── upload.js # File upload & batch processing
│ ├── results.js # Results rendering
│ ├── history.js # Processing history management
│ ├── stats.js # Statistics calculation
│ └── utils.js # Utility functions
├── tests/
│ ├── test_api.py # API integration tests
│ └── test_id_parser.py # ID parser unit tests
├── Dockerfile
└── docker-compose.yaml
```
## How It Works
### OCR Pipeline
1. **Image intake** — validates format and size, saves to a temporary directory
2. **Preprocessing** — auto-brightness correction (gamma), CLAHE contrast enhancement, bilateral noise filtering, intelligent resizing (800–2000px width)
3. **OCR pass 1** — standard preprocessing with EasyOCR
4. **Fallback passes** — if the ID number is not found, retries with aggressive preprocessing (sharpening + adaptive thresholding) and high-contrast preprocessing (histogram stretch + Otsu's thresholding)
5. **Field extraction** — spatial matching of bounding boxes to locate field values relative to labels
6. **Validation** — Luhn check on ID number, cross-validation of DOB/gender/citizenship against encoded values
7. **Cleanup** — temporary image is deleted immediately after processing
### OCR Error Correction
Common character confusions are automatically corrected during ID number extraction (e.g., `O`→`0`, `I`→`1`, `S`→`5`).
## SA ID Number Format
- Logs can be JSON or text format.
- `X-Request-ID` is accepted on requests and echoed in responses.
- Each request is logged with latency and request ID.
## Testing
```bash
python -m pytest tests/ -v
```