docs/spec_kit.md

Technical Specification Kit — PageParse

Any handwritten page → clean structured data. On CPU. Fully offline.

PageParse is designed for The CPU-First Hackathon to process unstructured handwriting, clean the image using computer vision, perform OCR, structure the text using an offline small language model, and persist it to SQLite — completely offline.

---

1. High-Level Architecture

                       ┌──────────────────────────────────────────────┐
                       │                  PageParse                    │
                       │            (100% on-device, CPU)              │
                       └──────────────────────────────────────────────┘

  scan / photo       OpenCV             ONNX / Tesseract     llama.cpp + GBNF      SQLite
  JPG · PNG · PDF  ┌──────────┐       ┌────────────────┐   ┌────────────────┐   ┌──────────┐
  ───────────────▶│ Ingestion│──────▶│  Preprocessing │──▶│  OCR (on CPU)  │──▶│ Transform│──▶│ Storage │
                  │          │ clean │  deskew·binarize│   │ TrOCR / Surya /│   │ SLM→JSON │   │  + vec  │
                  └──────────┘ denoise└────────────────┘   │   Tesseract    │   │ validated│   └──────────┘
                                                           └────────────────┘   └────────────────┘
                                                                                        │
                                                                              ┌─────────▼─────────┐
                                                                              │ query · search    │
                                                                              │ export · review   │
                                                                              └───────────────────┘

The system operates strictly in an offline-first/air-gapped mode. All models run locally on the CPU.

---

2. Ingestion & Preprocessing Specification

2.1 Ingestion

• Inputs: Scan or photograph of a handwritten page (.jpg, .jpeg, .png, .bmp, .tiff, .webp, .pdf).
• PWA/Web File Handler: FastAPI multipart upload endpoint.
• CLI File Handler: Recursively globbing directory files for batch processing.

2.2 OpenCV Preprocessing

To improve OCR transcription accuracy, raw inputs undergo:

1. Grayscale conversion: Simplifies the image channels.
2. Adaptive thresholding: Binarizes the image using Gaussian adaptive thresholding, coping with uneven lighting.
3. Denoising: Median filter to eliminate speckles and camera noise.
4. Deskewing: Calculates orientation of written text line bounding boxes and rotates back to horizontal.

---

3. Model & Runtime Specification

No GPU or CUDA execution providers are loaded. Model formats are standardized on ONNX and GGUF.

Stage	Task	Model	Runtime / Engine	Precision
OCR (Handwritten)	Local offline line recognition	TrOCR-small-handwritten	ONNX Runtime (CPU EP)	INT8
OCR (Printed)	OCR for structured clean documents	Tesseract LSTM	Tesseract Binaries	N/A
Transformation	Schema Mapping & Structuring	Qwen2.5-1.5B-Instruct	llama.cpp (llama-cpp-python)	Q4_K_M
Semantic Embeddings	Embedding generation for index	all-MiniLM-L6-v2	ONNX Runtime (CPU EP)	INT8

3.1 GBNF Grammar-Constrained Decoding

To ensure 100% compliance with the JSON output schema, the small language model (SLM) is constrained by a GBNF grammar (grammars/task.gbnf). This eliminates model hallucination, syntax errors, and missing fields.

---

4. Schema & Data Model Specifications

4.1 Input-Output JSON Schema

Handwriting is parsed and mapped to the following schema structure:

{
  "source_file": "string",
  "captured_date": "YYYY-MM-DD",
  "records": [
    {
      "type": "task | action_item | note | key_value | transcript_segment",
      "content": "string",
      "due_date": "YYYY-MM-DD | null",
      "priority": "high | medium | low | null",
      "category": "string | null",
      "speaker": "string | null",
      "timestamp": "string | null",
      "status": "todo | done | null",
      "confidence": 0.0
    }
  ]
}

4.2 Database Schema (SQLite)

CREATE TABLE IF NOT EXISTS sources (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    filename TEXT NOT NULL UNIQUE,
    source_type TEXT NOT NULL, -- image, audio, video, document
    image_path TEXT,
    cleaned_image_path TEXT,
    captured_date TEXT,
    raw_text TEXT NOT NULL,
    summary TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE IF NOT EXISTS records (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    source_id INTEGER NOT NULL,
    type TEXT NOT NULL,
    content TEXT NOT NULL,
    due_date TEXT,
    priority TEXT CHECK(priority IN ('high', 'medium', 'low') OR priority IS NULL),
    category TEXT,
    speaker TEXT,
    timestamp TEXT,
    status TEXT CHECK(status IN ('todo', 'done') OR status IS NULL),
    confidence REAL DEFAULT 1.0,
    user_edited INTEGER DEFAULT 0,
    FOREIGN KEY(source_id) REFERENCES sources(id) ON DELETE CASCADE
);

---

5. Telemetry & Observability Panel

• Process Telemetry: Tracks CPU percentage utilization and RAM usage (MB) during inference using psutil.
• System Telemetry: Visible on the web dashboard to demonstrate the exact physical load of the CPU-only model execution.

---

6. Offline Resiliency & Air-Gap Mode

• Air-gap Toggle: Command-line argument --airgap and Web UI switch that disables fallback external translate calls, proving total isolation.
• Local Fallback Translation: Includes static multi-language translation dictionaries for key hackathon demo texts (Hindi, Telugu, Tamil) so that translation features are testable offline.