Update README.md
Browse files
README.md
CHANGED
|
@@ -89,6 +89,23 @@ Synthetic checkbox crops generated from PDF answer-sheet templates using the scr
|
|
| 89 |
|
| 90 |
---
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
## Inference
|
| 93 |
|
| 94 |
### ONNX (recommended)
|
|
|
|
| 89 |
|
| 90 |
---
|
| 91 |
|
| 92 |
+
## Limitations & Known Issues
|
| 93 |
+
### Gray-ink misclassification
|
| 94 |
+
The training dataset uses synthetic marks with pixel grayscale values in the 10–80 range, simulating dark pen strokes. During real-world testing, lightly-marked checkboxes (pixel values ~128–200, typical of light gray ink or faint pencil strokes) fall outside this distribution and are frequently misclassified as empty.
|
| 95 |
+
|
| 96 |
+
An interim workaround — applying Otsu's binarization to normalize gray marks to black before inference — has been deployed in the Plom integration (plom/plom!3702), but this is a pre-processing patch rather than a model fix.
|
| 97 |
+
|
| 98 |
+
### Erased-mark ambiguity
|
| 99 |
+
Erased marks and lightly-filled marks occupy the same grayscale range (~128–200). The model currently has no mechanism to distinguish between:
|
| 100 |
+
|
| 101 |
+
A checkbox lightly filled with gray ink (should be filled)
|
| 102 |
+
A checkbox that was marked and then erased (should be empty)
|
| 103 |
+
This is an open problem: solving it likely requires either multi-threshold classification (e.g., a three-class filled / erased / empty output) or additional context such as smear patterns or residual graphite texture.
|
| 104 |
+
|
| 105 |
+
### Future Work
|
| 106 |
+
Retrain on a wider grayscale range: Expand the synthetic data generation to include filled marks across the full 10–200 pixel intensity range to handle gray ink natively without Otsu pre-processing (tracked in plom/plom#4277).
|
| 107 |
+
Erased mark handling: Investigate whether texture features or a three-class head (filled / erased / empty) can separate light fills from erasure residue.
|
| 108 |
+
|
| 109 |
## Inference
|
| 110 |
|
| 111 |
### ONNX (recommended)
|