docs/IMPLEMENTATION_PLAN.md

# Implementation Plan

# Zero-Shot Pattern Detection for Technical BOM Drawings

## 1. Current Assumptions

- The input is a pair of images: one cropped pattern image and one larger technical drawing image.
- The target drawings are mostly black-and-white or grayscale BOM/technical drawings with thin line symbols.
- The first version should run on CPU and be simple enough for HuggingFace Spaces.
- The system must be truly zero-shot: it must not hardcode symbol classes or require training when a new pattern is uploaded.
- The initial implementation will prioritize correctness, explainability, and a working demo over complex model-based methods.

## 2. Proposed Project Structure

```text
data/
  examples/
    pattern_*.png
    drawing_*.png
src/
  preprocess.py
  matcher.py
  nms.py
  visualize.py
  pipeline.py
app.py
requirements.txt
README.md
design.md
```

`data/` is reserved for test images and demo examples. The current workspace already contains this folder, but no uploaded image files were available in the attachments at the time this plan was written.

## 3. Implementation Flow

### Step 1: Store Example Data

Save reviewer/demo images under `data/examples/`.

Expected layout:

```text
data/examples/
  example1_pattern.png
  example1_drawing.png
  example2_pattern.png
  example2_drawing.png
  example3_pattern.png
  example3_drawing.png
```

Why:

- The SRS requires 2-3 example inputs for quick demo testing.
- Keeping examples under `data/` makes the Gradio app and README easier to reproduce.
- Reviewers can run the same cases locally and on HuggingFace.

Verification:

- Confirm image files exist under `data/examples/`.
- Open at least one pair and verify the pattern belongs to the drawing.

### Step 2: Preprocessing

Implement `src/preprocess.py`.

Responsibilities:

- Convert images to grayscale.
- Optionally apply binary thresholding.
- Optionally apply edge extraction.
- Preserve the original drawing image for visualization.

Why:

- Technical drawings are line-based, so grayscale/binary/edge representations reduce irrelevant color or scan variation.
- Template matching works better when the pattern and drawing are normalized into the same visual representation.

How this addresses requirements:

- Satisfies SRS FR-03.
- Improves robustness against scanned or noisy drawings.

Verification:

- Run preprocessing on sample images.
- Confirm output dimensions are valid.
- Confirm binary/edge modes do not destroy the visible symbol structure.

### Step 3: Template Matching

Implement `src/matcher.py` using OpenCV `cv2.matchTemplate`.

Default method:

- Use normalized correlation matching, such as `cv2.TM_CCOEFF_NORMED`.
- Slide the query pattern over the processed drawing.
- Collect locations whose score is above the configured threshold.

Why:

- This is directly aligned with zero-shot template matching.
- The uploaded pattern itself defines the search target.
- No training data, class labels, or fine-tuning are needed.
- It is CPU-friendly and easy to explain in the design document.

How this addresses requirements:

- Satisfies zero-shot detection.
- Returns multiple candidate locations.
- Produces confidence-like scores from the similarity map.

Verification:

- Run a known pattern/drawing pair.
- Confirm output includes candidate boxes with scores between 0 and 1.

### Step 4: Multi-Scale Search

Extend matching to test resized versions of the pattern.

Default scales:

```python
[0.75, 0.85, 1.0, 1.15, 1.3]
```

Why:

- In real drawings, the same symbol may appear slightly larger or smaller than the uploaded crop.
- Multi-scale matching handles this without training.

How this addresses requirements:

- Satisfies SRS FR-07 and the bonus requirement for scale variation.
- Keeps the method explainable and CPU-friendly if the scale list is small.

Verification:

- Confirm detections include the scale value.
- Confirm invalid scales are skipped when the resized pattern is larger than the drawing.

### Step 5: Optional Rotation Search

Add rotation search for default angles:

```python
[0, 90, 180, 270]
```

Why:

- Technical symbols may appear rotated in drawings.
- Right-angle rotation covers common schematic variations while keeping runtime controlled.

How this addresses requirements:

- Satisfies SRS FR-08 and the bonus requirement for rotation support.
- Avoids expensive dense angle search in the MVP.

Verification:

- Confirm detections include the angle value.
- Test with at least one rotated pattern if data is available.

### Step 6: Non-Maximum Suppression

Implement `src/nms.py`.

Responsibilities:

- Compute IoU between candidate boxes.
- Keep the highest-scoring box among heavily overlapping boxes.
- Use a default IoU threshold around `0.3`.

Why:

- Template matching often produces many nearby high-score locations around the same real object.
- NMS converts noisy candidate maps into clean final detections.

How this addresses requirements:

- Satisfies SRS FR-09.
- Reduces duplicate bounding boxes and improves reviewer readability.

Verification:

- Create overlapping candidate boxes in a small unit test.
- Confirm only the highest-score box is retained.

### Step 7: Visualization

Implement `src/visualize.py`.

Responsibilities:

- Draw bounding boxes on the original drawing image.
- Display confidence score near each box.
- Return the annotated image for the demo UI.

Why:

- Reviewers need to visually verify whether detections are correct.
- Visualization is required by the HuggingFace demo spec.

How this addresses requirements:

- Satisfies SRS FR-10.
- Makes false positives and missed detections easy to inspect.

Verification:

- Run the pipeline and inspect the output image.
- Confirm boxes do not hide the original drawing too heavily.

### Step 8: Pipeline Orchestration

Implement `src/pipeline.py`.

Responsibilities:

- Validate inputs.
- Run preprocessing.
- Run matching across selected scales and angles.
- Apply NMS.
- Format final JSON result.
- Generate visualization image.

Why:

- Keeps `app.py` simple.
- Makes the core detector reusable for CLI tests, unit tests, and Gradio.

How this addresses requirements:

- Connects all functional requirements into one inference flow.
- Produces both image and JSON outputs.

Verification:

- Run one function call with `pattern`, `drawing`, `threshold`, `scales`, and `angles`.
- Confirm it returns:
  - annotated image
  - `detections`
  - `count`
  - elapsed runtime if implemented

### Step 9: Gradio Demo

Implement `app.py`.

UI components:

- Pattern image upload.
- Drawing image upload.
- Threshold slider.
- Multi-scale checkbox.
- Rotation checkbox.
- Run button.
- Output image.
- JSON/text output.
- Example inputs from `data/examples/`.

Why:

- HuggingFace Spaces supports Gradio well.
- It satisfies the demo requirement with minimal infrastructure.

How this addresses requirements:

- Satisfies SRS section 2.3 and delivery requirement 4.2.
- Allows public reviewer testing.

Verification:

- Run `python app.py` locally.
- Upload example images.
- Confirm visualization and JSON are displayed.

### Step 10: Documentation

Create:

- `README.md`: install, run locally, run demo, example usage.
- `design.md`: architecture, module explanation, strengths, limitations, future work.
- `requirements.txt`: Python dependencies.

Why:

- The assignment explicitly requires GitHub repo documentation and a system design document.
- A clear explanation matters because the assessment prioritizes zero-shot reasoning and method choice.

Verification:

- Follow README from a clean environment.
- Confirm the documented command starts the demo.

## 4. Why This Method Is the First Choice

OpenCV template matching is the best MVP choice for this assignment because:

- It is naturally zero-shot: the query image is the template.
- It does not require a labeled dataset.
- It is easy to explain and debug.
- It runs on CPU.
- It fits black-and-white line drawings better than many generic object detectors.
- It gives direct similarity scores that can be used as confidence values.
- It can detect multiple repeated instances in one drawing.

Foundation models such as Grounding DINO or SAM are not the first choice for this MVP because:

- They are heavier for CPU HuggingFace deployment.
- They are not naturally driven by arbitrary cropped technical-symbol image queries.
- They may perform poorly on thin black-and-white schematic symbols without adaptation.
- They add complexity that is not necessary for the first working solution.

Feature matching methods such as ORB/SIFT can be added later as a fallback, but they are not the first implementation because:

- Small schematic symbols may not provide enough stable keypoints.
- Homography-based matching can fail on very simple line patterns.
- Template matching is simpler and more predictable for cropped symbol search.

## 5. Known Risks and Mitigations

| Risk | Impact | Mitigation |
| --- | --- | --- |
| Very simple patterns match unrelated lines | False positives | Use higher threshold, NMS, edge/binary mode, optional contour filtering later |
| Scale or rotation differs strongly | Missed detections | Multi-scale and right-angle rotation search |
| Large drawings are slow | Runtime issue | Limit scale/angle search, resize large drawings if needed, add image pyramid optimization later |
| Pattern crop includes too much whitespace | Lower confidence | Add optional crop-to-content preprocessing |
| Scan noise or blur | Lower confidence / false positives | Binary thresholding, denoising, edge mode |

## 6. Verification Plan

Minimum checks before submission:

1. Confirm `data/examples/` contains at least 2-3 pattern/drawing pairs.
2. Run local inference for each example.
3. Confirm JSON output has this shape:

```json
{
  "detections": [
    {
      "bbox": [120, 240, 32, 45],
      "confidence": 0.92,
      "scale": 1.0,
      "angle": 0
    }
  ],
  "count": 1
}
```

4. Confirm visualization image shows boxes around matched regions.
5. Confirm typical CPU runtime is under 60 seconds per request.
6. Confirm README commands work.

## 7. Next Implementation Step

The next concrete step is to place the actual test images into `data/examples/`. After that, implementation should start with the smallest end-to-end vertical slice:

1. Load one pattern and one drawing.
2. Run grayscale template matching at scale `1.0`, angle `0`.
3. Return bbox JSON.
4. Draw bbox visualization.
5. Wrap it in Gradio.

Only after that baseline works should multi-scale, rotation, and preprocessing options be added.