---
license: openrail++
language:
- en
- zh
library_name: diffusers
base_model:
- sd2-community/stable-diffusion-2-inpainting
- diffusers/stable-diffusion-xl-1.0-inpainting-0.1
tags:
- stable-diffusion
- stable-diffusion-xl
- lora
- inpainting
- industrial
---

# DefectForge VisA Inpainting LoRAs

Per-object SD2 and SDXL inpainting LoRA adapters trained from 10 real anomalous VisA
training images for each of `pcb1` and `capsules`.

**繁中摘要：**每個物件只使用 10 張真實瑕疵訓練圖，分別訓練 SD2／SDXL
inpainting LoRA。權重必須搭配 ROI crop、binary mask 與 blend-back 流程使用；直接對
整張 AOI 影像做一般文字生圖，不是本模型的支援用法。

## Files and base locks

```text
lora_sd2/
  pcb1/final/
  capsules/final/
lora_sdxl/
  pcb1/final/
  capsules/final/
```

Each final directory includes the UNet adapter, learned token embedding adapter(s),
tokenizer state, and final training configuration.

| Family | Base repository | Immutable revision | Resolution |
|---|---|---|---:|
| SD2 | `sd2-community/stable-diffusion-2-inpainting` | `5f74973cbb64c8568780732c17f43eb269d63a0d` | 512 |
| SDXL | `diffusers/stable-diffusion-xl-1.0-inpainting-0.1` | `115134f363124c53c7d878647567d04daf26e41e` | 1024 |

## Trigger tokens

The tokens are frozen unsupervised pseudo-types, not official VisA labels.

| Object | Token | Frozen training-mask components |
|---|---|---:|
| pcb1 | `<pcb1-type0>` | 16 |
| pcb1 | `<pcb1-type1>` | 7 |
| capsules | `<capsules-type0>` | 9 |
| capsules | `<capsules-type1>` | 3 |

## Supported inference path

Use the repository pipeline so inference crops around the placed defect mask, runs the
matching base model and adapter at its native resolution, then blends the generated ROI
back into the original image. This preserves the surrounding AOI frame.

```powershell
uv run python src/synthetic/generate_diffusion.py `
  --config configs/generate_sd2.yaml `
  --object pcb1 `
  --n 1 `
  --refine
```

The equivalent Python flow is:

```python
# 1. Load a frozen normal image and a placement mask.
# 2. Expand the mask bounding box by the config crop_ratio.
# 3. Resize that ROI + mask to the base model's native resolution.
# 4. Inpaint with the object-specific LoRA and trigger token.
# 5. Resize the generated ROI back and feather/Poisson blend only inside the mask.
#
# The versioned implementation and exact metadata contract live in:
# src/synthetic/generate_diffusion.py
```

Do not bypass crop-to-ROI and blend-back by sending a full-resolution industrial frame
directly to a generic text-to-image call; that is outside this release's tested
interface.

## Training data and leakage boundary

- Each object uses 10 real anomalous images selected with seed 42.
- Selections are published in `splits/fewshot_selection.json` in the GitHub repository.
- The single test partition is VisA `2cls_highshot` test.
- No frozen test image, mask, or embedding is read during adapter training or generation.
- Every test image/mask SHA-256 is published in `splits/test_blocklist.json`.

## Limitations

- Ten training images per object create a high risk of overfitting and limited defect
  diversity.
- Trigger tokens are pseudo-types and may mix multiple visual failure mechanisms.
- SDXL `pcb1-type0` can hallucinate component- or insect-like shapes, while capsules
  `type0` can repeatedly produce jewellery-, button-, lens-, or mechanical-ring-like
  objects; search improves boundary fit but does not guarantee semantic correctness.
- Inpainting quality is sensitive to mask geometry, crop ratio, guidance scale, and
  background domain.
- These adapters cover only `pcb1` and `capsules`; they are not general industrial
  anomaly models.
- Outputs remain subject to the base models' Open RAIL++-M use restrictions.

## License chain

Source Code 與第三方 Artifact 的完整界線見
[THIRD_PARTY_NOTICES.md](https://github.com/kuotunyu/defectforge-visa-synthetic-data/blob/main/THIRD_PARTY_NOTICES.md)。

<!-- BEGIN VERIFIED LICENSE_CHAIN -->
| 資產 | License | DefectForge 義務 |
|---|---|---|
| VisA 原始 Dataset | CC BY 4.0 | 標示 VisA 與其論文；Hugging Face Dataset 不得包含原始影像 |
| `sd2-community/stable-diffusion-2-inpainting` | CreativeML Open RAIL++-M | 保留用途限制，並揭露 preservation mirror |
| `diffusers/stable-diffusion-xl-1.0-inpainting-0.1` | CreativeML Open RAIL++-M | 保留用途限制 |
| `facebook/dinov2-base` | Apache-2.0 | 標示模型與 DINOv2 論文 |
| DefectForge Synthetic Images | CC BY 4.0 | 視為 VisA 衍生內容；保留 VisA attribution，並揭露 Diffusion base model License |
| DefectForge LoRA Weights | CreativeML Open RAIL++-M | 繼承對應 base model 的限制，並附上 License 連結 |
| DefectForge Source Code | MIT | MIT 僅授權程式碼，不包含 Dataset 與 Model Weights |
<!-- END VERIFIED LICENSE_CHAIN -->

## Citation and provenance

Methodology, checksums, training reports, and limitations are published in
[kuotunyu/defectforge-visa-synthetic-data](https://github.com/kuotunyu/defectforge-visa-synthetic-data).
GitHub Citation Metadata 見
[CITATION.cff](https://github.com/kuotunyu/defectforge-visa-synthetic-data/blob/main/CITATION.cff).
The project is an independent open-source replication and is not affiliated with or
endorsed by NVIDIA.