OMR Checkbox Classifier

A lightweight binary CNN that classifies scanned exam answer-sheet checkboxes as filled or empty. Trained on synthetic crops generated from PDF answer-sheet templates, with augmentation to simulate real scan noise, skew, and marking styles.

Developed for the Plom Project — an open-source platform for automated grading of paper exams.

Developer: Chenfeng Xu, Undergraduate Research Assistant, UBC
Supervisors: Prof. Andrew Rechnitzer & Prof. Colin B. MacDonald
License: AGPL-3.0 (to comply with Plom Project requirements)

Model Details

Property	Value
Architecture	SmallOMRNet (custom lightweight CNN)
Input	1 × 64 × 64 grayscale image
Output	Scalar sigmoid probability of checkbox being filled
Task	Binary classification: `filled` / `empty`
Export formats	ONNX (`omr_model.onnx`)
Parameters	~120k

Architecture

SmallOMRNet uses depthwise separable convolutions (DSConv) for efficiency. Each DSConv block is a depthwise conv + BN + ReLU followed by a pointwise conv + BN + ReLU.

Input: (1, 64, 64)
→ Conv2d(1→24, 3×3, stride=2) + BN + ReLU     # (24, 32, 32)
→ DSConv(24→32)                                 # (32, 32, 32)
→ DSConv(32→48, stride=2)                       # (48, 16, 16)
→ DSConv(48→64)                                 # (64, 16, 16)
→ DSConv(64→96, stride=2)                       # (96,  8,  8)
→ DSConv(96→128)                                # (128, 8,  8)
→ AdaptiveAvgPool2d(1) → Flatten                # (128,)
→ Linear(128→64) + ReLU + Dropout(0.20)
→ Linear(64→1)                                  # logit
→ Sigmoid                                       # fill probability ∈ [0, 1]

Training

Dataset

Synthetic checkbox crops generated from PDF answer-sheet templates using the scripts in this repository. Each crop is a padded region around a single checkbox (pad ratio 0.35), rendered at 300 DPI from the template PDF.

Classes: filled (label 1) / empty (label 0)
Image size: 64 × 64 grayscale
Normalisation: mean=0.5, std=0.5

Augmentation (training only)

Transform	Parameters
RandomAffine	rotate ±8°, translate ±6%, scale 92–108%, shear ±5°
RandomPerspective	distortion 0.18, p=0.25
ColorJitter	brightness 0.15, contrast 0.15
GaussianBlur	kernel 3, σ ∈ [0.1, 1.0]
RandomErasing	p=0.10, scale 2–10%

Hyperparameters

Parameter	Value
Optimizer	AdamW
Learning rate	0.001
Weight decay	0.0001
LR schedule	CosineAnnealingLR
Loss	BCEWithLogitsLoss (class-balanced pos_weight)
Batch size	128
Max epochs	20
Early stopping	patience=5, monitor=ROC-AUC
Random seed	42

Inference

ONNX (recommended)

import cv2
import numpy as np
import onnxruntime as ort

IMG_SIZE = 64
MEAN, STD = 0.5, 0.5

session = ort.InferenceSession("omr_model.onnx", providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name

def predict(crop_bgr: np.ndarray) -> float:
    """Return fill probability for a single BGR checkbox crop."""
    gray = cv2.cvtColor(crop_bgr, cv2.COLOR_BGR2GRAY)
    resized = cv2.resize(gray, (IMG_SIZE, IMG_SIZE), interpolation=cv2.INTER_LINEAR)
    arr = (resized.astype(np.float32) / 255.0 - MEAN) / STD
    batch = arr[np.newaxis, np.newaxis, :, :]          # (1, 1, 64, 64)
    logit = session.run(None, {input_name: batch})[0]
    return float(1.0 / (1.0 + np.exp(-logit)))        # sigmoid

crop = cv2.imread("path/to/checkbox_crop.png")
prob = predict(crop)
print("filled" if prob >= 0.5 else "empty", f"  prob={prob:.3f}")

Decision thresholds

Parameter	Default	Env var override
Fill threshold	0.50	`OMR_THRESHOLD`
Uncertain margin	0.10	`OMR_UNCERTAIN_MARGIN`
Min confidence	0.55	`OMR_MIN_CONFIDENCE`

A prediction is flagged uncertain when |prob - threshold| < uncertain_margin or confidence < min_confidence, where confidence = 1 - normalised_entropy(prob).

Repository Contents

Path	Description
`omr_model.onnx`	Trained model — primary deployment artifact
`config/config.yaml`	Full pipeline config: template variants, detector settings, training hyperparameters
`data/raw/template/`	Answer-sheet template PDFs (6 variants: 2-/4-/6-choice, mixed, assembled, math quiz)
`data/template_map/`	JSON coordinate maps for each template variant

Template Variants

Six answer-sheet layouts are supported. Each variant has a corresponding template PDF and a pre-built template map (checkbox coordinate file).

Variant	Choices per question	Template PDF
`2choice`	A, B	`template_2choice.pdf`
`4choice`	A, B, C, D	`template_4choice.pdf`
`6choice`	A, B, C, D, E, F	`template_6choice.pdf`
`mixed`	variable	`template_mixed.pdf`
`assembled_4choice`	A, B, C, D	`template_4choice_assembled.pdf`
`math_quiz`	A, B, C, D	`exam_math_quiz.pdf`

Intended Use

Designed for automated grading pipelines that pre-crop answer-sheet checkboxes (after scan alignment and template matching) and need a fast, CPU-friendly classifier to decide filled vs. empty. Intended to run as part of the plom-digit-recognition-server.

Out of scope: general object detection, handwriting recognition, forms with non-checkbox mark types.

Downloads last month: -; Downloads are not tracked for this model. How to track