Jwalit commited on
Commit
4d3fefd
·
verified ·
1 Parent(s): 0962d87

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +43 -59
README.md CHANGED
@@ -14,62 +14,57 @@ metrics:
14
  - precision
15
  - recall
16
  pipeline_tag: image-classification
17
- model-index:
18
- - name: document-moire-detector
19
- results:
20
- - task:
21
- type: image-classification
22
- name: Moiré Pattern Detection
23
- metrics:
24
- - type: accuracy
25
- value: 0.9950
26
- name: Test Accuracy
27
- - type: f1
28
- value: 0.9950
29
- name: Test F1
30
  ---
31
 
32
  # Document Moiré Detection Model (V2)
33
 
34
- A fine-tuned **DeiT-small** (Vision Transformer, 22M params) model for detecting moiré patterns in document images.
35
 
36
  ## Model Description
37
 
38
- This model performs binary classification to detect whether a document image contains moiré patterns —
39
- visual artifacts that commonly occur when:
40
- - Photographing a screen displaying a document
41
- - Scanning documents at certain resolutions
42
- - Screen-capturing documents with resolution mismatches
43
 
44
  **Labels:**
45
- - `clean` (0): No moiré patterns detected
46
  - `moire` (1): Moiré patterns detected
47
 
48
- ## Training
49
-
50
- - **Base model:** `facebook/deit-small-patch16-224` (22M parameters)
51
- - **Training data:** 8,000 samples (4,000 clean + 4,000 synthetic moiré)
52
- - **Source images:** [rvl-cdip document classification dataset](https://huggingface.co/datasets/hf-tuner/rvl-cdip-document-classification)
53
- - **Moiré generation:** 6 synthetic methods:
54
- 1. Resize aliasing (screen-camera simulation)
55
- 2. Frequency-domain pattern overlay
56
- 3. Multi-frequency band interference with color fringing
57
- 4. Screen pixel grid + capture simulation
58
- 5. **Subtle moiré** — low-strength single-frequency patterns (hard examples)
59
- 6. **Localized moiré** partial-image patterns with gaussian mask
60
- - **Epochs:** 5
61
- - **Learning rate:** 3e-5 (cosine schedule)
62
- - **Effective batch size:** 64
63
- - **Label smoothing:** 0.05
 
 
 
 
 
 
 
 
 
 
 
64
 
65
  ## Performance
66
 
67
- | Metric | Validation | Test (held-out) |
68
- |-----------|-----------|-----------------|
69
- | Accuracy | 98.5% | 99.5% |
70
- | F1 Score | 0.985 | 0.995 |
71
- | Precision | 98.2% | 99.3% |
72
- | Recall | 98.8% | 99.7% |
73
 
74
  ## Usage
75
 
@@ -77,13 +72,12 @@ visual artifacts that commonly occur when:
77
  from transformers import pipeline
78
 
79
  classifier = pipeline("image-classification", model="Jwalit/document-moire-detector")
80
- result = classifier("path/to/document_image.jpg")
81
  print(result)
82
  # [{'label': 'clean', 'score': 0.99}, {'label': 'moire', 'score': 0.01}]
83
  ```
84
 
85
  Or manually:
86
-
87
  ```python
88
  from transformers import AutoImageProcessor, AutoModelForImageClassification
89
  from PIL import Image
@@ -94,23 +88,13 @@ model = AutoModelForImageClassification.from_pretrained("Jwalit/document-moire-d
94
 
95
  image = Image.open("document.jpg")
96
  inputs = processor(image, return_tensors="pt")
97
-
98
  with torch.no_grad():
99
  logits = model(**inputs).logits
100
- predicted_class = logits.argmax(-1).item()
101
-
102
- print(model.config.id2label[predicted_class]) # 'clean' or 'moire'
103
  ```
104
 
105
- ## Version History
106
-
107
- | Version | Model | Train Size | Methods | Val F1 | Test F1 |
108
- |---------|-------|-----------|---------|--------|---------|
109
- | V1 | DeiT-tiny (5.5M) | 6,000 | 4 | 0.998 | 0.995 |
110
- | **V2** | **DeiT-small (22M)** | **8,000** | **6** | **0.985** | **0.995** |
111
-
112
  ## Limitations
113
-
114
- - Trained on synthetic moiré patterns may not capture all real-world moiré variations
115
- - Optimized for document images; performance on natural scene images may vary
116
- - Input images are resized to 224×224; very subtle moiré in high-resolution images may be lost
 
14
  - precision
15
  - recall
16
  pipeline_tag: image-classification
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ---
18
 
19
  # Document Moiré Detection Model (V2)
20
 
21
+ A fine-tuned **DeiT-small** Vision Transformer for detecting moiré patterns in document images.
22
 
23
  ## Model Description
24
 
25
+ Binary classifier: detects whether a document image contains moiré artifacts
26
+ (common from screen photography, scanning, or screen captures).
 
 
 
27
 
28
  **Labels:**
29
+ - `clean` (0): No moiré patterns
30
  - `moire` (1): Moiré patterns detected
31
 
32
+ ## V2 Improvements (over V1)
33
+ - **Larger model:** DeiT-small (22M params) vs DeiT-tiny (5.5M)
34
+ - **More training data:** 8,000 samples vs 6,000
35
+ - **6 moiré methods** (added subtle + localized patterns as hard examples)
36
+ - **Label smoothing** (0.05) for better calibration
37
+ - **Stronger augmentation** (rotation, hue jitter)
38
+
39
+ ## Training Details
40
+
41
+ | Parameter | Value |
42
+ |-----------|-------|
43
+ | Base model | `facebook/deit-small-patch16-224` |
44
+ | Parameters | 22M |
45
+ | Training samples | 8,000 (4,000 clean + 4,000 moiré) |
46
+ | Test samples | 800 (400 clean + 400 moiré) |
47
+ | Epochs | 5 |
48
+ | Learning rate | 3e-05 (cosine schedule) |
49
+ | Effective batch size | 64 |
50
+ | Label smoothing | 0.05 |
51
+
52
+ ### Moiré Generation Methods
53
+ 1. **Resize aliasing** — downscale+upscale with NEAREST interpolation + pattern overlay
54
+ 2. **Pattern overlay** — sinusoidal interference with per-channel color variation
55
+ 3. **Multi-frequency** — 2-4 patterns at different frequencies + color displacement
56
+ 4. **Screen simulation** — pixel grid + rotation + moiré overlay
57
+ 5. **Subtle moiré** — very low strength single-frequency (hard examples)
58
+ 6. **Localized moiré** — moiré in elliptical region with gaussian mask
59
 
60
  ## Performance
61
 
62
+ | Metric | Eval |
63
+ |--------|------|
64
+ | Accuracy | 0.9912 |
65
+ | F1 Score | 0.9913 |
66
+ | Precision | 0.9852 |
67
+ | Recall | 0.9975 |
68
 
69
  ## Usage
70
 
 
72
  from transformers import pipeline
73
 
74
  classifier = pipeline("image-classification", model="Jwalit/document-moire-detector")
75
+ result = classifier("path/to/document.jpg")
76
  print(result)
77
  # [{'label': 'clean', 'score': 0.99}, {'label': 'moire', 'score': 0.01}]
78
  ```
79
 
80
  Or manually:
 
81
  ```python
82
  from transformers import AutoImageProcessor, AutoModelForImageClassification
83
  from PIL import Image
 
88
 
89
  image = Image.open("document.jpg")
90
  inputs = processor(image, return_tensors="pt")
 
91
  with torch.no_grad():
92
  logits = model(**inputs).logits
93
+ label = model.config.id2label[logits.argmax(-1).item()]
94
+ print(label) # 'clean' or 'moire'
 
95
  ```
96
 
 
 
 
 
 
 
 
97
  ## Limitations
98
+ - Trained on synthetic moiré — may not capture all real-world variations
99
+ - Optimized for document images; natural scenes may vary
100
+ - Input resized to 224×224; very subtle moiré in high-res images may be lost