vlm_recall_boost (default: follows use_vlm):
- Widens the probe-focused scale sweep ([0.55..1.55] vs [0.80..1.20]) when VLM is on.
- Lowers the DINOv2 gate to 0.80 and relaxes Chamfer/H-Chamfer filters, letting
more genuinely-present instances survive to Stage 3 for VLM adjudication.
vlm_reject_only=True (new default):
- Blacklist mode: drops a candidate ONLY when the VLM confidently names a concretely-
different component (inductor/capacitor/diode/op-amp/crystal/logic-gate).
- Candidates labelled transistor/other/wire-junction (_UNTRUSTED_LABELS) are KEPT:
the 2B model defaults ambiguous line-art to 'transistor', so a strict whitelist
wrongly discards genuine high-conf resistors. Blacklist preserves them.
Also adds four diagnostic comparison scripts for evaluating VLM filter modes.
Paths updated to use drawings/ folder (image-cache paths were ephemeral).
Add a local Vision-Language Model as an optional third verification stage to
reject false positives that NCC + DINOv2 cannot separate (inductor/crystal/
op-amp/diode share low-level structure with resistors).
- vlm_verifier.py: VLMVerifier with open-classification (not yes/no, which a
2B model answers 'yes' to indiscriminately). Stays zero-shot: the template's
own VLM class defines the target. Lazy-loads ~4.5GB weights on first use.
- pipeline.py: use_vlm flag (default False), vlm_keep_min_conf=0.75 so only
borderline candidates are judged; high-conf TPs are shielded from the 2B
model's mislabelling. Runs after structural filters, before final NMS.
- web UI: 'Use VLM' toggle (index.html/app.js/css), server.py runtime flag.
- requirements.txt: optional transformers/accelerate/qwen-vl-utils.
- model_survey.md: full A/B findings (yes/no 40/40 -> classify 7/33; on user
schematic 25 -> 13 borderline-only vs 25 -> 6 over-aggressive).
- scripts/compare_vlm.py: off/on A/B harness.
…
Complete the new verification layer for simple-outline templates.
- dino_dense_matcher.py: DINOv2 dense sliding-window matcher with a fast
NCC scale probe. Activates as Pass C in detect_auto only when the pattern
appears larger than the template (probe_s > 1.10) — the regime NCC's
[0.30-1.0] scale grid structurally cannot reach.
- pipeline.py: gate Pass C behind a new `use_dino_dense` config flag
(default True; preserves the validated 10/10 behaviour).
- dino_verifier.py: embed_crops_normalized() for orientation-invariant
prototype/dense embeddings.
- Remove clip_verifier.py: model_survey benchmark showed CLIP image-to-image
gives no FP separation over DINOv2, and text-guided mode breaks zero-shot.
- model_survey.md: record the CLIP finding and the DINODense A/B result
(ON==OFF on all 2 examples + 6 real drawings; dense is a dormant
scale-invariant fallback for the untested larger-than-template case).
- scripts/eval_all.py, eval_drawings.py: A/B evaluation harnesses.
…
feat: Zero-shot pattern detection pipeline for engineering BOM drawings
Duy
7d835bb
Pipeline: NCC multi-scale (Stage 1) + DINOv2 ViT-S/14 (Stage 2) + structural filters
- Auto-classify templates as simple (outline, e.g. resistor) or complex (e.g. bridge rectifier)
- Simple path: Chamfer pre-filter + hybrid DINOv2 bypass for borderline candidates
- Complex path: dual micro-pass (0deg + 90deg) with DINOv2 derotate=True
- FastAPI web UI with PaceOS-style dashboard
- HuggingFace Spaces Docker deployment ready
Results: Drawing 1 (resistor) 10 detections, Drawing 4 (BR) 4 detections,
Example 1 10 detections, Example 2 4 detections. All 10 unit tests pass.
