mgmbiik/image-context-sorter-dinov2

Image Context Sorter — DINOv2 partial fine-tune

Restoration-work photo classifier for 하우스캐치 (water-damage restoration). Classifies a single photo into one of 8 work-type / stage categories.

Private / internal — trained on client work photos; not for redistribution.

Results (locked test set, 8 classes)

Model	macro-F1	top-2
518px frozen linear probe (baseline)	0.601	0.830
224px frozen linear probe (control)	0.617	0.800
224px partial fine-tune (this model)	0.737	0.895

Every class improved over the probe; the previously weakest class (곰팡이제거) rose 0.44 → 0.67 and its 곰팡이제거↔누수탐지 confusion was eliminated.

Model

• Base: vit_base_patch14_dinov2.lvd142m (timm / Meta DINOv2)
• Fine-tune: partial — the last 4 transformer blocks + final norm + a new 8-class head are unfrozen; the rest of the backbone is frozen. Input 224px (position-embedding interpolation). Discriminative LR (backbone 1e-5 / head 1e-3), AdamW + cosine schedule, float16 autocast (Apple MPS), class-weighted cross-entropy, early stopping on val macro-F1. No vertical-flip augmentation (ceiling/floor structure is a label cue).
• Calibration: temperature scaling, T ≈ 2.44 (fit on the val split). Apply it to logits before softmax for honest confidence; it does not change argmax/accuracy.

Classes (index order)

0 철거중 · 1 비포1 · 2 애프터 · 3 보양작업 · 4 석고보드설치 · 5 누수탐지 · 6 도배작업 · 7 곰팡이제거

How to load

import timm, torch
from torchvision import transforms as T

model = timm.create_model(
    "vit_base_patch14_dinov2.lvd142m", pretrained=False, num_classes=8, img_size=224
)
model.load_state_dict(torch.load("finetune.pt", map_location="cpu"))
model.eval()

tf = T.Compose([
    T.Resize(224), T.CenterCrop(224), T.ToTensor(),
    T.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
# logits = model(tf(img).unsqueeze(0))
# probs  = torch.softmax(logits / 2.44, dim=1)   # temperature-calibrated

License

This is a derivative of Meta's DINOv2 (vit_base_patch14_dinov2.lvd142m). Use is governed by the DINOv2 model license — see https://github.com/facebookresearch/dinov2. Confirm redistribution / commercial terms before any external distribution.

Notes

• Weights are a raw PyTorch state_dict (finetune.pt). Consider converting to safetensors for safer, code-free loading.
• See config.json for the exact class list, temperature, and preprocessing.